LLMKube
LLMKube is a Kubernetes operator that runs self-hosted large language models (LLMs) on local hardware.
Platform availability
Available on 4 of 10 platforms
- Unraid (not listed)
- TrueNAS (not listed)
- Umbrel (not listed)
- ZimaOS (not listed)
- Proxmox (not listed)
- Helm (not listed)
- Mac
- Windows
- Linux
- Any server
Health score?A 0-100 score computed nightly from four factors: maintained (40%: recent commits and steady activity), popular (25%: GitHub stars, log scale), easy to install (20%: platforms, install notes, setup guides, Docker image) and light to run (15%: minimum RAM, ARM support). When the source code has been checked, a fifth factor, security check (20%: automated scan for committed secrets, unsafe desktop-app settings, known-vulnerable dependencies and project hygiene; any serious finding caps it at 20), is added. Factors without data are left out and the rest are rescaled.
75/100 Good
- Maintained
- 100last commit 3 days ago
- Popular
- 49267 GitHub stars
- Easy to install
- 554 install platforms, install notes
- Light to run
- No data
- Security check
- 76no serious findings, 4 minor, CI, tests
Based on 4 of 5 factors.
Availability: platform listings confirmed today
- GitHub stars
- 267
- Open issues
- 139
- Last commit
- 2026-11-18
- Latest release
- v0.10.19
- License
- Apache-2.0
- Activity
- rising
Checked 3 days ago - source: GitHub (defilantech/LLMKube)
Resources & Compatibility
Requirements not published yet.
Checked 3 days ago - source: Docker Hub image tags and published docs
First-install notes
- Install cluster tools first Install Minikube (the small local Kubernetes cluster that LLMKube can run in), kubectl (the command tool for the cluster), and Helm (the cluster app installer) using the commands for your OS. Without these, you cannot start the cluster or install LLMKube. source
- Start Minikube with enough space Start Minikube with the recommended 4 CPUs, 6GB memory, and 20GB disk, then run minikube status until the cluster says ready. Too little memory or disk can make model containers crash or prevent LLMKube from running workloads reliably. source
- Install LLMKube and check the controller Add the LLMKube Helm repository and run helm install llmkube llmkube/llmkube --namespace llmkube-system --create-namespace, then use kubectl get pods -n llmkube-system to confirm the controller pod is Running. If this manager is not Running, your Model and InferenceService objects will not be processed, so LLMKube will not start the models. source
- Apple Silicon On Apple Silicon, LLMKube runs inference as native macOS processes instead of containers. source
Fetched 3 days ago - each note links to its source
Variants
Alternatives (4)
- KubeAI
- NVIDIA NIM - NVIDIA NIM is a set of accelerated inference microservices that allow organizations to run AI models on NVIDIA GPUs anywhere.
- Ollama - Ollama is a local runtime for large language models (LLMs) that provides an application programming interface (API) and command-line interf…
- llm-d
Checked 3 days ago - source: AlternativeTo
Common questions
How do I add an Apple Silicon Mac to LLMKube without exposing cluster credentials?
Use the out-of-cluster Metal Agent. Create a one-use join token from the cluster with llmkube metal token create, paste the printed join string on the Mac with llmkube metal join, and deploy models with accelerator: metal. The join string expires in 30 minutes and the Mac does not receive cluster credentials.
What Minikube memory and disk size should I use before installing LLMKube?
Start Minikube with about 4 CPUs, 6GB of memory, and 20GB of disk, then wait for minikube status to say ready. Too little memory or disk can crash model containers or stop LLMKube from running workloads reliably.
Where can LLMKube download models from, and are they stored locally?
LLMKube can download model files from Hugging Face, S3, OCI, or a URL. It caches them, so the same model is available locally after the initial download.
For a Metal-backed inference service, where should a reverse proxy point?
Point it at the ordinary Kubernetes Service created by LLMKube for that InferenceService. The documented example calls that service on port 8080 at the OpenAI-compatible chat completions endpoint.
Can I rely on the LLMKube CRD fields remaining stable?
LLMKube is pre-1.0, and every CRD is v1alpha1. Fields can change between minor releases.
How are off-cluster agents kept updated?
LLMKube uses declarative, health-gated self-update for off-cluster agents. The edge paths mentioned are helm and brew.
Answers sourced from dev.to, github.com, github.com, llmkube.com
Community
- LLMKube – A Kubernetes operator for local LLMs across Nvidia and Mac fleets (forum)
- Show HN: LLMKube – Kubernetes for Local LLMs with GPU Acceleration (forum)
- LLMKube: A Kubernetes LLM Inference Operator for Production ... (forum)
Collected 3 days ago - public community threads