LLM Harbor
LLM Harbor is a local large language model (LLM) management toolkit that unifies llama.cpp, vLLM, and Ollama behind a single CLI on macOS, Windows, and Linux.
Platform availability
Available on 4 of 10 platforms
- Unraid (not listed)
- TrueNAS (not listed)
- Umbrel (not listed)
- ZimaOS (not listed)
- Proxmox (not listed)
- Helm (not listed)
- Mac
- Windows
- Linux
- Any server
Health score?A 0-100 score computed nightly from four factors: maintained (40%: recent commits and steady activity), popular (25%: GitHub stars, log scale), easy to install (20%: platforms, install notes, setup guides, Docker image) and light to run (15%: minimum RAM, ARM support). When the source code has been checked, a fifth factor, security check (20%: automated scan for committed secrets, unsafe desktop-app settings, known-vulnerable dependencies and project hygiene; any serious finding caps it at 20), is added. Factors without data are left out and the rest are rescaled.
70/100 Good
- Maintained
- 100last commit 10 days ago
- Popular
- 703,280 GitHub stars
- Easy to install
- 554 install platforms, install notes
- Light to run
- 108 GB minimum RAM
- Security check
- 70no serious findings, 21 minor, CI, tests
Availability: platform listings confirmed today
- GitHub stars
- 3,280
- Open issues
- 7
- Last commit
- 2026-11-11
- Latest release
- v0.5.20
- License
- Apache-2.0
- Activity
- declining
Checked 3 days ago - source: GitHub (av/harbor)
Resources & Compatibility
Minimum requirements: 8 GB RAM, 10 GB disk.
Checked 3 days ago - source: Docker Hub image tags and published docs
First-install notes
- Install the terminal tool with cargo If you want the command-line tool, install Rust first and run `cargo install llm`, where cargo is Rust's installer. Without these steps, the `llm` command will not be available in your terminal. source
- Set your AI provider key first Before using LLM Harbor, set the API key for the AI service you plan to use, such as OPENAI_API_KEY in an environment variable (a named secret value the program reads). Without that key, LLM Harbor cannot call the cloud AI service, so chats and requests fail with authentication errors. source
- API server uses port 3000 If you run the server example, it listens at 127.0.0.1:3000 with an auth key, meaning your own computer can reach it but other devices usually cannot. Open port 3000 only when another device must use the server, and keep the auth key on because it helps protect your AI setup from uninvited users. source
- Installing a desktop build For macOS, Windows, and Linux, download the latest native build from the GitHub releases page as an alternative to building from source with cargo. source
Fetched 3 days ago - each note links to its source
Variants
Alternatives (8)
- Agenta - The open-source LLMOps platform: prompt playground, prompt management, LLM evaluation, and LLM observability all in one place.
- Khoj - Khoj is an open-source AI personal assistant and second brain.
- Local Deep Research - Local Deep Research is a privacy-focused artificial intelligence (AI) research assistant that performs systematic research on complex quest…
- LocalAI - LocalAI is a free, open-source, self-hosted AI engine that provides a drop-in, OpenAI-compatible API for running local models.
- Ollama - Ollama is a local runtime for large language models (LLMs) that provides an application programming interface (API) and command-line interf…
- ZeroClaw - claw done right 🦀.
- anythingllm - AnythingLLM is a local, all-in-one AI app for self-hosters that works with any large language model, providing document chat, retrieval-aug…
- llama.cpp - Llama.cpp is a pure C/C++ inference engine for large language models, including Meta's LLaMA.
Checked 3 days ago - source: AlternativeTo
Common questions
Do I need to build LLM Harbor from source?
No; for macOS, Windows, and Linux, you can download the latest native build from the releases page instead of building from source with cargo.
What credential do I need to set before using the app?
Set the API key for the provider you plan to use, such as OPENAI_API_KEY, in an environment variable. Without it, chats and requests can fail with authentication errors.
Which port should I expose if another device needs the API server?
The server example listens on 127.0.0.1:3000, so other devices usually cannot reach it. Open port 3000 only when another device must use the server, and keep the auth key enabled.
Do I need to write configuration files to wire up the local stack?
It launches a fully pre-wired local LLM stack with backends, frontends, and supporting services without writing any configuration.
Answers sourced from github.com, github.com, www.zendot.org
Community
- Show HN: Harbor v0.4.19 – harbor launch –back end vLLM –web codex (forum)
- Show HN: A single CLI to manage llama.cpp/vLLM/Ollama models (forum)
- Show HN: Harbor – Containerized LLM Toolkit (forum)
Collected 3 days ago - public community threads