# LLM Harbor

LLM Harbor is a local large language model (LLM) management toolkit that unifies llama.cpp, vLLM, and Ollama behind a single CLI on macOS, Windows, and Linux.

Canonical page: https://ourlocalhost.com/app/8579/llm-harbor

## Description

LLM Harbor is a local large language model (LLM) management toolkit that unifies llama.cpp, vLLM, and Ollama behind a single CLI on macOS, Windows, and Linux. It launches and models-manages containerized backends and exposes them through an API server, so homelab users can serve and switch between local models without juggling separate tools and CLIs.

## Platform availability

Available on 4 of 10 platforms: Mac, Windows, Linux, Any server.

## Desktop and other devices

Mac, Windows, Linux.

## Health score

70/100 (Good), based on 5 of 5 factors.
- Maintained: 100 (last commit 10 days ago)
- Popular: 70 (3,280 GitHub stars)
- Easy to install: 55 (4 install platforms, install notes)
- Light to run: 10 (8 GB minimum RAM)
- Security check: 70 (no serious findings, 21 minor, CI, tests)

## Links

- GitHub: https://github.com/av/harbor
- desktop: https://github.com/av/harbor/releases/latest

## Common questions

### Do I need to build LLM Harbor from source?

No; for macOS, Windows, and Linux, you can download the latest native build from the releases page instead of building from source with cargo.

### What credential do I need to set before using the app?

Set the API key for the provider you plan to use, such as OPENAI_API_KEY, in an environment variable. Without it, chats and requests can fail with authentication errors.

### Which port should I expose if another device needs the API server?

The server example listens on 127.0.0.1:3000, so other devices usually cannot reach it. Open port 3000 only when another device must use the server, and keep the auth key enabled.

### Do I need to write configuration files to wire up the local stack?

It launches a fully pre-wired local LLM stack with backends, frontends, and supporting services without writing any configuration.

## Alternatives

- Agenta: The open-source LLMOps platform: prompt playground, prompt management, LLM evaluation, and LLM observability all in one place.
- Khoj: Khoj is an open-source AI personal assistant and second brain.
- Local Deep Research: Local Deep Research is a privacy-focused artificial intelligence (AI) research assistant that performs systematic research on complex quest…
- LocalAI: LocalAI is a free, open-source, self-hosted AI engine that provides a drop-in, OpenAI-compatible API for running local models.
- Ollama: Ollama is a local runtime for large language models (LLMs) that provides an application programming interface (API) and command-line interf…
- ZeroClaw: claw done right 🦀.
- anythingllm: AnythingLLM is a local, all-in-one AI app for self-hosters that works with any large language model, providing document chat, retrieval-aug…
- llama.cpp: Llama.cpp is a pure C/C++ inference engine for large language models, including Meta's LLaMA.

---
Source: Our Local Host (https://ourlocalhost.com/app/8579/llm-harbor), the self-hosted and open-source app directory.
