inferzero
Local inference server for Intel Arc GPUs using Intel's Level Zero and OpenCL runtime.
Platform availability
Available on 1 of 10 platforms
- Unraid
- TrueNAS (not listed)
- Umbrel (not listed)
- ZimaOS (not listed)
- Proxmox (not listed)
- Helm (not listed)
- Mac (not listed)
- Windows (not listed)
- Linux (not listed)
- Any server (not listed)
Health score?A 0-100 score computed nightly from four factors: maintained (40%: recent commits and steady activity), popular (25%: GitHub stars, log scale), easy to install (20%: platforms, install notes, setup guides, Docker image) and light to run (15%: minimum RAM, ARM support). When the source code has been checked, a fifth factor, security check (20%: automated scan for committed secrets, unsafe desktop-app settings, known-vulnerable dependencies and project hygiene; any serious finding caps it at 20), is added. Factors without data are left out and the rest are rescaled.
63/100 Good
- Maintained
- 84last commit 0 days ago
- Popular
- 3874 GitHub stars
- Easy to install
- 201 install platform, Docker image
- Light to run
- No data
- Security check
- 95no serious findings, CI, tests
Based on 4 of 5 factors.
Availability: platform listings confirmed today
- GitHub stars
- 74
- Open issues
- 35
- Last commit
- 2026-11-21
- Latest release
- 2026.11.22.03
- License
- Apache-2.0
- Activity
- rising
Checked today - source: GitHub (rizagp/inferzero)
Resources & Compatibility
Requirements not published yet.
Checked today - source: Docker Hub image tags and published docs
First-install notes
No first-install notes yet.
Variants
Alternatives
No alternatives collected yet.
Common questions
Can inferzero be used with existing OpenAI, Anthropic, or Ollama clients?
Yes. It serves an OpenAI-compatible, Anthropic-compatible, and Ollama-compatible API, and it includes a web UI.
Can inferzero run mixture-of-experts models that are larger than GPU memory?
Yes. It streams experts from the SSD into a GPU cache while the GPU computes, allowing MoE models larger than the GPU's memory to run.
Can inferzero serve two concurrent Pi coding-agent sessions on one model?
Yes. It can run two Pi sessions at once on one model with --slots 2, and when both are writing, their next-token work runs in one engine pass.
Was inferzero tested on an Arc B580, and what environment did that report use?
A user reported tests for an Arc B580 12 GB in a PCIe 3.0 slot using NixOS 26.05, kernel 7.3.0, intel-compute-runtime 26.18.38308.1, Level Zero loader 1.28.5, and inferzero commit 7703141.
If an A770 inference run is much slower than other reports, is the model necessarily the cause?
No. A report says the model is not the cause, because two other A770s achieved about 30-38 tokens per second on the 4B and 9B models while one was much slower.
Answers sourced from github.com, github.com, github.com
Community
No community discussions collected yet.