Llama-swap

Llama-swap is a lightweight, zero-dependency tool that lets you run multiple generative AI models on your machine and hot-swap between them on demand.

  • AI
  • Other
  • Productivity
  • Tools & Utilities
Llama-swap is a lightweight, zero-dependency tool that lets you run multiple generative AI models on your machine and hot-swap between them on demand. It works with any OpenAI- and Anthropic-API-compatible server, making it an easy way to power local AI workflows.

Platform availability

Available on 1 of 10 platforms

  • Unraid
  • TrueNAS (not listed)
  • Umbrel (not listed)
  • ZimaOS (not listed)
  • Proxmox (not listed)
  • Helm (not listed)
  • Mac (not listed)
  • Windows (not listed)
  • Linux (not listed)
  • Any server (not listed)

Health score?A 0-100 score computed nightly from four factors: maintained (40%: recent commits and steady activity), popular (25%: GitHub stars, log scale), easy to install (20%: platforms, install notes, setup guides, Docker image) and light to run (15%: minimum RAM, ARM support). When the source code has been checked, a fifth factor, security check (20%: automated scan for committed secrets, unsafe desktop-app settings, known-vulnerable dependencies and project hygiene; any serious finding caps it at 20), is added. Factors without data are left out and the rest are rescaled.

72/100 Good

Maintained
100last commit 8 days ago
Popular
766,252 GitHub stars
Easy to install
351 install platform, install notes, Docker image
Light to run
108 GB minimum RAM
Security check
94no serious findings, 1 minor, CI, tests

Availability: platform listings confirmed today, links checked 21 days ago

GitHub stars
6,252
Open issues
116
Last commit
2026-11-12
Latest release
v272
License
MIT
Activity
steady

Checked 7 days ago - source: GitHub (mostlygeek/llama-swap)

Resources & Compatibility

Minimum requirements: 8 GB RAM.

Checked 54 days ago - source: Docker Hub image tags and published docs

First-install notes

  • Start the WebUI first Run `llamafactory-cli webui` after installation to open the browser interface. This is a good first check because if it does not open, you probably do not have a working app yet. source
  • Set port and GPU before API If you use the API, start it with `API_PORT=8000` and `CUDA_VISIBLE_DEVICES=0` (the NVIDIA GPU number you want to use). If the port or GPU setting is wrong, clients cannot reach the model or the job may run on the wrong hardware. source
  • NPU users: install CANN first For Ascend NPU hardware, install the CANN Toolkit before trying training or inference, and make sure you have enough disk space. Without this toolkit, the NPU commands cannot run at all. source

Fetched 53 days ago - each note links to its source

Alternatives

No alternatives collected yet.

Common questions

Can I front multiple local model servers without replacing my existing OpenAI or Anthropic clients?

Yes. llama-swap acts as a reverse proxy and supports any OpenAI and Anthropic API compatible server, so clients can send requests to it while it hot-swaps backends.

Does llama-swap include a local helper for configuration questions?

Yes. The Help page in the sidebar is an agent that calls llama-swap documentation tools and answers configuration questions using config.example.yaml and the knowledge base, running entirely on a local model.

Where should I look when a model process does not start or misbehaves?

Use the common issues catalog, which covers the lifecycle of the Process entity, configuration pitfalls, and environmental conflicts.

Can it manage several LLMs on one server without manually starting and stopping them?

Yes. Llama-Swap is positioned to handle starting/stopping different models so you can switch between them on a single server.

Answers sourced from deepwiki.com, github.com, selfhost.directory, www.kdnuggets.com

Community

Collected 49 days ago - public community threads