# KoboldCpp

KoboldCpp is a lightweight, self-contained AI backend for text and image generation running GGML and GGUF models on CPU or GPU, bundled with the KoboldAI Lite frontend for a simple local AI experience. It is open source and self-hosted, installable on Docker Hub and Unraid.

Canonical page: https://ourlocalhost.com/app/1391/koboldcpp

## Description

KoboldCpp is a lightweight, self-contained AI backend for text and image generation running GGML and GGUF models on CPU or GPU, bundled with the KoboldAI Lite frontend for a simple local AI experience.

## Platform availability

Available on 1 of 9 platforms: Unraid.

## Health score

19/100 (Low), based on 4 of 4 factors.
- Maintained: 16 (last commit 570 days ago)
- Popular: 17 (6 GitHub stars)
- Easy to install: 35 (1 install platform, install notes, Docker image)
- Light to run: 10 (8 GB minimum RAM)

## Links

- Docker Hub: https://hub.docker.com/r/koboldai/koboldcpp/
- GitHub: https://github.com/oromis995/UnraidKoboldCpp
- Unraid: https://ca.unraid.net/apps/koboldcpp-0tlkpqd1b9pvbd?q=koboldcpp
- Official website: https://koboldai.org

## Common questions

### Why will KoboldCpp not start on Linux after downloading the binary?

Run chmod +x on the downloaded Linux file so the computer can execute it. Without that permission step, starting KoboldCpp is blocked.

### Does KoboldCpp need a model file before the server is started?

Yes. Start the server with a .gguf model file or a folder of model files. Without a model file, the server has nothing to answer with.

### Why does KoboldCpp not use an NVIDIA GPU when installed with Nix?

Enable CUDA in your Nix configuration and list your GPU architecture code, such as sm_75. Without that setting, KoboldCpp won't use the GPU and may be slow or fail.

### Is a batch size of 1024 a problem in KoboldCpp?

A 1024 batch size is not a problem and is actually recommended for performance if you have the memory.

### Does KoboldCpp provide API endpoints for other applications?

Yes. KoboldCpp provides multiple compatible API endpoints for popular web services.

---
Source: Our Local Host (https://ourlocalhost.com/app/1391/koboldcpp), the self-hosted and open-source app directory.