# LiteLLM

LiteLLM is a self-hosted AI gateway and proxy server that lets teams manage authentication, load balancing, routing, and spend tracking for 100+ large language models (LLMs) using the OpenAI format. It is open source and self-hosted, installable on Proxmox, Kubernetes (Helm) and Unraid.

Canonical page: https://ourlocalhost.com/app/2990/litellm

## Description

LiteLLM is a self-hosted AI gateway and proxy server that lets teams manage authentication, load balancing, routing, and spend tracking for 100+ large language models (LLMs) using the OpenAI format.

## Platform availability

Available on 5 of 10 platforms: Unraid, TrueNAS, Proxmox, Helm, Any server.

## Health score

92/100 (Excellent), based on 4 of 4 factors.
- Maintained: 100 (last commit 7 days ago)
- Popular: 96 (63,254 GitHub stars)
- Easy to install: 75 (5 install platforms, install notes, Docker image)
- Light to run: 85 (512 MB minimum RAM)

## Links

- Proxmox: https://community-scripts.github.io/ProxmoxVE/scripts?id=litellm
- Kubernetes (Helm): https://artifacthub.io/packages/helm/litellm/litellm-helm
- Unraid: https://ca.unraid.net/apps/litellm-07y1pp31ua6hn7?q=LiteLLM
- Official website: https://docs.litellm.ai/docs/
- Documentation: https://docs.litellm.ai/docs/simple_proxy
- GitHub: https://github.com/berriai/litellm

## Common questions

### What address should client apps use to reach the LiteLLM proxy?

LiteLLM normally listens on 0.0.0.0:4000, so set the app base URL to that address. If port 4000 is blocked or the address is wrong, apps cannot reach the proxy.

### How should I start LiteLLM with model names and provider keys?

Create a config.yaml file with your model names and provider keys, then start the proxy with litellm --config config.yaml --detailed_debug. Missing settings can stop model calls from being routed or authenticated.

### Which environment variables do I set before starting LiteLLM if I use managed API keys or a database?

Set DATABASE_URL and LITELLM_MASTER_KEY before starting LiteLLM. Missing values can let the server start but cause key setup or authenticated requests to fail.

### What backing stores did a production LiteLLM Proxy setup use?

The described production setup uses Redis and PostgreSQL to back the LiteLLM Proxy while routing traffic across multiple providers.

### Where does LiteLLM put text response content in its consistent output?

Text responses are available at response['choices'][0]['message']['content']. Common exceptions are mapped to OpenAI-style exceptions.

## Alternatives

- 9Router: 9Router is an OpenAI-compatible AI router that connects coding tools such as Claude Code, Cursor, and Codex to more than 40 AI providers th…
- Experiential: Open-source, OpenAI-compatible gateway that routes hosted, BYOK, local, and custom models with access controls, budgets, and usage attribut…
- OmniRoute: OmniRoute is an open-source AI gateway for homelab users that provides a drop-in OpenAI-compatible endpoint across 352+ LLM providers, with…
- OpenRouter: OpenRouter is an AI model aggregator service that provides developers and users access to a wide range of large language models and generat…

---
Source: Our Local Host (https://ourlocalhost.com/app/2990/litellm), the self-hosted and open-source app directory.
