# Basalt

Basalt is a fast local inference engine that runs large language models on a single NVIDIA Blackwell GPU, keeping the parts of a model that do not fit in graphics memory in system RAM. It is open source and self-hosted, installable on Unraid.

Canonical page: https://ourlocalhost.com/app/8918/basalt

## Description

Basalt is a fast local inference engine that runs large language models on a single NVIDIA Blackwell GPU, keeping the parts of a model that do not fit in graphics memory in system RAM. It supports Qwen3.8-Flash-Next, DeepSeek-V4.1-Flash and the dense Qwen3.8-27B, and serves an OpenAI- and Anthropic-compatible API with a live web UI that shows per-request speed. This template runs an image built from Basalt's own Dockerfile. It is for homelab users with an RTX 50-series card who want fast local model inference without a multi-GPU setup.

## Platform availability

Available on 1 of 10 platforms: Unraid.

## Health score

53/100 (Fair), based on 5 of 5 factors.
- Maintained: 91 (last commit 2 days ago)
- Popular: 63 (1,486 GitHub stars)
- Easy to install: 35 (1 install platform, install notes, Docker image)
- Light to run: 30 (4 GB minimum RAM)
- Security check: 0 (5 serious findings, 22 minor, CI, tests)

## Security check

5 serious findings: Uses pillow 12.3.0, which has a known critical vulnerability; Uses pillow 12.3.0, which has a known critical vulnerability; Uses pillow 12.3.0, which has a known critical vulnerability; Uses pillow 12.3.0, which has a known critical vulnerability; Uses pillow 12.3.0, which has a known critical vulnerability.
Automated checks on commit 1771f9d, 19 Nov 2026. Not an audit.

## Links

- Unraid: https://ca.unraid.net/apps/basalt-0r61ijd06qrrot?q=Basalt
- Official website: https://basalt.jgulfo.com

## Common questions

### How do I add a model to Basalt?

A model is a single .basalt file. Download it into the Models folder from the terminal.

### What client APIs does Basalt expose?

It serves an OpenAI-compatible API and an Anthropic-compatible API, along with a chat page.

### Where can model experts be kept when they do not fit in VRAM?

They can be kept in system RAM, and optionally on more GPUs.

### What should I preserve when troubleshooting a failed stage?

Preserve the complete command, stdout, stderr, Basalt_log.txt, Basalt_checkpoint.txt, software versions, database configuration, and input checksums.

---
Source: Our Local Host (https://ourlocalhost.com/app/8918/basalt), the self-hosted and open-source app directory.
