# LLMKube

LLMKube is a Kubernetes operator that runs self-hosted large language models (LLMs) on local hardware.

Canonical page: https://ourlocalhost.com/app/8580/llmkube

## Description

LLMKube is a Kubernetes operator that runs self-hosted large language models (LLMs) on local hardware. It exposes OpenAI-compatible endpoints for NVIDIA, AMD, Intel, and Apple Silicon hosts from a single configuration spec. It suits platform teams that need consistent model deployment across mixed or air-gapped GPU fleets.

## Platform availability

Available on 4 of 10 platforms: Mac, Windows, Linux, Any server.

## Desktop and other devices

Mac, Windows, Linux.

## Health score

75/100 (Good), based on 4 of 5 factors.
- Maintained: 100 (last commit 3 days ago)
- Popular: 49 (267 GitHub stars)
- Easy to install: 55 (4 install platforms, install notes)
- Security check: 76 (no serious findings, 4 minor, CI, tests)

## Links

- Official website: https://llmkube.com
- GitHub: https://github.com/defilantech/LLMKube
- desktop: https://github.com/defilantech/LLMKube/releases/latest
- Documentation: https://llmkube.com/docs

## Common questions

### How do I add an Apple Silicon Mac to LLMKube without exposing cluster credentials?

Use the out-of-cluster Metal Agent. Create a one-use join token from the cluster with llmkube metal token create, paste the printed join string on the Mac with llmkube metal join, and deploy models with accelerator: metal. The join string expires in 30 minutes and the Mac does not receive cluster credentials.

### What Minikube memory and disk size should I use before installing LLMKube?

Start Minikube with about 4 CPUs, 6GB of memory, and 20GB of disk, then wait for minikube status to say ready. Too little memory or disk can crash model containers or stop LLMKube from running workloads reliably.

### Where can LLMKube download models from, and are they stored locally?

LLMKube can download model files from Hugging Face, S3, OCI, or a URL. It caches them, so the same model is available locally after the initial download.

### For a Metal-backed inference service, where should a reverse proxy point?

Point it at the ordinary Kubernetes Service created by LLMKube for that InferenceService. The documented example calls that service on port 8080 at the OpenAI-compatible chat completions endpoint.

### Can I rely on the LLMKube CRD fields remaining stable?

LLMKube is pre-1.0, and every CRD is v1alpha1. Fields can change between minor releases.

### How are off-cluster agents kept updated?

LLMKube uses declarative, health-gated self-update for off-cluster agents. The edge paths mentioned are helm and brew.

## Alternatives

- KubeAI
- NVIDIA NIM: NVIDIA NIM is a set of accelerated inference microservices that allow organizations to run AI models on NVIDIA GPUs anywhere.
- Ollama: Ollama is a local runtime for large language models (LLMs) that provides an application programming interface (API) and command-line interf…
- llm-d

---
Source: Our Local Host (https://ourlocalhost.com/app/8580/llmkube), the self-hosted and open-source app directory.
