LiteLLM
LiteLLM is a self-hosted AI gateway and proxy server that lets teams manage authentication, load balancing, routing, and spend tracking for 100+ large…
Platform availability
Available on 5 of 10 platforms
- Unraid
- TrueNAS
- Umbrel (not listed)
- ZimaOS (not listed)
- Proxmox
- Helm
- Mac (not listed)
- Windows (not listed)
- Linux (not listed)
- Any server
Health score?A 0-100 score computed nightly from four factors: maintained (40%: recent commits and steady activity), popular (25%: GitHub stars, log scale), easy to install (20%: platforms, install notes, setup guides, Docker image) and light to run (15%: minimum RAM, ARM support). When the source code has been checked, a fifth factor, security check (20%: automated scan for committed secrets, unsafe desktop-app settings, known-vulnerable dependencies and project hygiene; any serious finding caps it at 20), is added. Factors without data are left out and the rest are rescaled.
92/100 Excellent
- Maintained
- 100last commit 7 days ago
- Popular
- 9663,254 GitHub stars
- Easy to install
- 755 install platforms, install notes, Docker image
- Light to run
- 85512 MB minimum RAM
Availability: platform listings confirmed today, links checked 3 days ago
- GitHub stars
- 63,254
- Open issues
- 5189
- Last commit
- 2026-11-14
- Latest release
- v1.109.1
- Activity
- declining
Checked 7 days ago - source: GitHub (BerriAI/litellm)
Resources & Compatibility
Minimum requirements: 512 MB RAM.
Checked 54 days ago - source: Docker Hub image tags and published docs
First-install notes
- Start with your settings file Create a config.yaml text file with your model names and provider keys. Start the LiteLLM proxy (the server that forwards AI requests) with litellm --config config.yaml --detailed_debug, because missing settings can stop model calls from being routed or authenticated. source
- Keep port 4000 reachable LiteLLM normally listens on http://0.0.0.0:4000, so set your apps' base URL to that address. If port 4000 is blocked or the address is wrong, your apps cannot reach the proxy. source
- Set database and master key If you use LiteLLM-managed API keys or a database, set DATABASE_URL and LITELLM_MASTER_KEY (the main admin key) before starting LiteLLM. Missing values can let the server start but cause key setup or authenticated requests to fail. source
Fetched 53 days ago - each note links to its source
Variants
Alternatives (4)
- 9Router - 9Router is an OpenAI-compatible AI router that connects coding tools such as Claude Code, Cursor, and Codex to more than 40 AI providers th…
- Experiential - Open-source, OpenAI-compatible gateway that routes hosted, BYOK, local, and custom models with access controls, budgets, and usage attribut…
- OmniRoute - OmniRoute is an open-source AI gateway for homelab users that provides a drop-in OpenAI-compatible endpoint across 352+ LLM providers, with…
- OpenRouter - OpenRouter is an AI model aggregator service that provides developers and users access to a wide range of large language models and generat…
Checked 54 days ago - source: AlternativeTo
Common questions
What address should client apps use to reach the LiteLLM proxy?
LiteLLM normally listens on 0.0.0.0:4000, so set the app base URL to that address. If port 4000 is blocked or the address is wrong, apps cannot reach the proxy.
How should I start LiteLLM with model names and provider keys?
Create a config.yaml file with your model names and provider keys, then start the proxy with litellm --config config.yaml --detailed_debug. Missing settings can stop model calls from being routed or authenticated.
Which environment variables do I set before starting LiteLLM if I use managed API keys or a database?
Set DATABASE_URL and LITELLM_MASTER_KEY before starting LiteLLM. Missing values can let the server start but cause key setup or authenticated requests to fail.
What backing stores did a production LiteLLM Proxy setup use?
The described production setup uses Redis and PostgreSQL to back the LiteLLM Proxy while routing traffic across multiple providers.
Where does LiteLLM put text response content in its consistent output?
Text responses are available at response['choices'][0]['message']['content']. Common exceptions are mapped to OpenAI-style exceptions.
Answers sourced from dev.to, docs.litellm.ai, docs.litellm.ai, docs.litellm.ai, www.reddit.com
Community
- Reddit (reddit)
- Lost in LLM Land? Let LiteLLM Be Your Guide to Navigate the ... - Reddit (reddit)
- conduct of litellm as an open source projects - Reddit (reddit)
- Lost in LLM Land? LiteLLM is Your Guide! - Reddit (reddit)
- Newest 'litellm' Questions - Stack Overflow (forum)
Collected 49 days ago - public community threads