# inferzero

inferzero is a self-hosted ai app. Local inference server for Intel Arc GPUs using Intel's Level Zero and OpenCL runtime. It is open source and self-hosted, installable on Unraid.

Canonical page: https://ourlocalhost.com/app/9018/inferzero

## Description

Local inference server for Intel Arc GPUs using Intel's Level Zero and OpenCL runtime. This template uses a container image built and published from the reproducible Dockerfile in the catduck templates repository; it is not the locally built Uranium image. Install the required Unraid kernel separately: download the matching custom kernel release, follow its release instructions, and install it manually on the host before enabling GPU passthrough. Docker and Community Applications never install or modify the host kernel. No public kernel release link is available yet; do not install this template for GPU use until the matching release and its manual install and rollback instructions are published. Setup: set Model to a catalog key (default 4b) or a path to a GGUF file in the Models folder. Models download from Hugging Face into that persistent folder; model files can be several GB or

## Platform availability

Available on 1 of 10 platforms: Unraid.

## Health score

63/100 (Good), based on 4 of 5 factors.
- Maintained: 84 (last commit 0 days ago)
- Popular: 38 (74 GitHub stars)
- Easy to install: 20 (1 install platform, Docker image)
- Security check: 95 (no serious findings, CI, tests)

## Links

- Unraid: https://ca.unraid.net/apps/app_86d996402d94f9a7
- GitHub: https://github.com/rizagp/inferzero
- Official website: https://rizagp.github.io/inferzero/

## Common questions

### Can inferzero be used with existing OpenAI, Anthropic, or Ollama clients?

Yes. It serves an OpenAI-compatible, Anthropic-compatible, and Ollama-compatible API, and it includes a web UI.

### Can inferzero run mixture-of-experts models that are larger than GPU memory?

Yes. It streams experts from the SSD into a GPU cache while the GPU computes, allowing MoE models larger than the GPU's memory to run.

### Can inferzero serve two concurrent Pi coding-agent sessions on one model?

Yes. It can run two Pi sessions at once on one model with --slots 2, and when both are writing, their next-token work runs in one engine pass.

### Was inferzero tested on an Arc B580, and what environment did that report use?

A user reported tests for an Arc B580 12 GB in a PCIe 3.0 slot using NixOS 26.05, kernel 7.3.0, intel-compute-runtime 26.18.38308.1, Level Zero loader 1.28.5, and inferzero commit 7703141.

### If an A770 inference run is much slower than other reports, is the model necessarily the cause?

No. A report says the model is not the cause, because two other A770s achieved about 30-38 tokens per second on the 4B and 9B models while one was much slower.

---
Source: Our Local Host (https://ourlocalhost.com/app/9018/inferzero), the self-hosted and open-source app directory.
