AI Gateway

A high-performance, multi-provider LLM proxy supporting 31 AI providers, 8 routing strategies, and an extensible plugin pipeline.

  • AI
  • Development
  • Network Services
A high-performance, multi-provider LLM proxy supporting 31 AI providers, 8 routing strategies, and an extensible plugin pipeline.

Platform availability

Available on 1 of 10 platforms

  • Unraid (not listed)
  • TrueNAS (not listed)
  • Umbrel (not listed)
  • ZimaOS (not listed)
  • Proxmox (not listed)
  • Helm
  • Mac (not listed)
  • Windows (not listed)
  • Linux (not listed)
  • Any server (not listed)

Health score?A 0-100 score computed nightly from four factors: maintained (40%: recent commits and steady activity), popular (25%: GitHub stars, log scale), easy to install (20%: platforms, install notes, setup guides, Docker image) and light to run (15%: minimum RAM, ARM support). When the source code has been checked, a fifth factor, security check (20%: automated scan for committed secrets, unsafe desktop-app settings, known-vulnerable dependencies and project hygiene; any serious finding caps it at 20), is added. Factors without data are left out and the rest are rescaled.

68/100 Good

Maintained
95last commit 16 days ago
Popular
601,008 GitHub stars
Easy to install
251 install platform, install notes
Light to run
No data

Based on 3 of 4 factors.

Availability: platform listings confirmed today

GitHub stars
1,008
Open issues
84
Last commit
2026-11-05
License
MIT
Activity
declining

Checked 14 days ago - source: GitHub (Azure-Samples/AI-Gateway)

Resources & Compatibility

Requirements not published yet.

First-install notes

  • Getting started AI Gateway Labs is a sample repository for the Azure API Management GenAI gateway. The README points to the AI Gateway Dev Portal, a starter developer-portal template you can fork and customise (e.g. in VS Code) to build a portal on top of the gateway. source

Fetched 41 days ago - each note links to its source

Alternatives

No alternatives collected yet.

Common questions

Is rollback supported after updating the gateway?

Yes, the operations gateway supports updates and rollback.

What common issues should I troubleshoot when self-hosted model serving is slow or failing?

Check for GPU not detected, OOM model loads, slow tokens, missing models, and Ollama service failures.

Where should the gateway sit in a self-hosted LLM request flow?

A request comes in and hits the AI gateway, which enforces guardrails on the incoming request.

Can I use fallback behavior in the gateway?

The operations gateway supports fallback.

Answers sourced from d-central.tech, github.com, medium.com

Community

Collected 48 days ago - public community threads