inferzero

Local inference server for Intel Arc GPUs using Intel's Level Zero and OpenCL runtime.

  • AI
  • Productivity
  • Tools & Utilities
Local inference server for Intel Arc GPUs using Intel's Level Zero and OpenCL runtime. This template uses a container image built and published from the reproducible Dockerfile in the catduck templates repository; it is not the locally built Uranium image. Install the required Unraid kernel separately: download the matching custom kernel release, follow its release instructions, and install it manually on the host before enabling GPU passthrough. Docker and Community Applications never install or modify the host kernel. No public kernel release link is available yet; do not install this template for GPU use until the matching release and its manual install and rollback instructions are published. Setup: set Model to a catalog key (default 4b) or a path to a GGUF file in the Models folder. Models download from Hugging Face into that persistent folder; model files can be several GB or

Platform availability

Available on 1 of 10 platforms

  • Unraid
  • TrueNAS (not listed)
  • Umbrel (not listed)
  • ZimaOS (not listed)
  • Proxmox (not listed)
  • Helm (not listed)
  • Mac (not listed)
  • Windows (not listed)
  • Linux (not listed)
  • Any server (not listed)

Health score?A 0-100 score computed nightly from four factors: maintained (40%: recent commits and steady activity), popular (25%: GitHub stars, log scale), easy to install (20%: platforms, install notes, setup guides, Docker image) and light to run (15%: minimum RAM, ARM support). When the source code has been checked, a fifth factor, security check (20%: automated scan for committed secrets, unsafe desktop-app settings, known-vulnerable dependencies and project hygiene; any serious finding caps it at 20), is added. Factors without data are left out and the rest are rescaled.

63/100 Good

Maintained
84last commit 0 days ago
Popular
3874 GitHub stars
Easy to install
201 install platform, Docker image
Light to run
No data
Security check
95no serious findings, CI, tests

Based on 4 of 5 factors.

Availability: platform listings confirmed today

GitHub stars
74
Open issues
35
Last commit
2026-11-21
Latest release
2026.11.22.03
License
Apache-2.0
Activity
rising

Checked today - source: GitHub (rizagp/inferzero)

Resources & Compatibility

Requirements not published yet.

Checked today - source: Docker Hub image tags and published docs

First-install notes

No first-install notes yet.

Alternatives

No alternatives collected yet.

Common questions

Can inferzero be used with existing OpenAI, Anthropic, or Ollama clients?

Yes. It serves an OpenAI-compatible, Anthropic-compatible, and Ollama-compatible API, and it includes a web UI.

Can inferzero run mixture-of-experts models that are larger than GPU memory?

Yes. It streams experts from the SSD into a GPU cache while the GPU computes, allowing MoE models larger than the GPU's memory to run.

Can inferzero serve two concurrent Pi coding-agent sessions on one model?

Yes. It can run two Pi sessions at once on one model with --slots 2, and when both are writing, their next-token work runs in one engine pass.

Was inferzero tested on an Arc B580, and what environment did that report use?

A user reported tests for an Arc B580 12 GB in a PCIe 3.0 slot using NixOS 26.05, kernel 7.3.0, intel-compute-runtime 26.18.38308.1, Level Zero loader 1.28.5, and inferzero commit 7703141.

If an A770 inference run is much slower than other reports, is the model necessarily the cause?

No. A report says the model is not the cause, because two other A770s achieved about 30-38 tokens per second on the 4B and 9B models while one was much slower.

Answers sourced from github.com, github.com, github.com

Community

No community discussions collected yet.