# AI Gateway

AI Gateway is a self-hosted ai app. A high-performance, multi-provider LLM proxy supporting 31 AI providers, 8 routing strategies, and an extensible plugin pipeline. It is open source and self-hosted, installable on Kubernetes (Helm).

Canonical page: https://ourlocalhost.com/app/4919/ai-gateway

## Description

A high-performance, multi-provider LLM proxy supporting 31 AI providers, 8 routing strategies, and an extensible plugin pipeline.

## Platform availability

Available on 1 of 10 platforms: Helm.

## Health score

68/100 (Good), based on 3 of 4 factors.
- Maintained: 95 (last commit 16 days ago)
- Popular: 60 (1,008 GitHub stars)
- Easy to install: 25 (1 install platform, install notes)

## Links

- GitHub: https://github.com/Azure-Samples/AI-Gateway
- Documentation: https://learn.microsoft.com/azure/api-management/genai-gateway-capabilities
- Official website: https://aka.ms/ai-gateway/labs
- Kubernetes (Helm): https://artifacthub.io/packages/helm/ferro-labs/ai-gateway

## Common questions

### Is rollback supported after updating the gateway?

Yes, the operations gateway supports updates and rollback.

### What common issues should I troubleshoot when self-hosted model serving is slow or failing?

Check for GPU not detected, OOM model loads, slow tokens, missing models, and Ollama service failures.

### Where should the gateway sit in a self-hosted LLM request flow?

A request comes in and hits the AI gateway, which enforces guardrails on the incoming request.

### Can I use fallback behavior in the gateway?

The operations gateway supports fallback.

---
Source: Our Local Host (https://ourlocalhost.com/app/4919/ai-gateway), the self-hosted and open-source app directory.
