Intel IPEX-LLM Ollama

Intel IPEX-LLM Ollama is a containerized Ollama build with Intel IPEX-LLM acceleration that lets AI enthusiasts run large language models (LLMs) quickly on…

  • AI
Intel IPEX-LLM Ollama is a containerized Ollama build with Intel IPEX-LLM acceleration that lets AI enthusiasts run large language models (LLMs) quickly on Intel GPUs, NPUs, and CPUs, though the upstream IPEX-LLM project is archived and carries known security issues.

Platform availability

Available on 1 of 10 platforms

  • Unraid
  • TrueNAS (not listed)
  • Umbrel (not listed)
  • ZimaOS (not listed)
  • Proxmox (not listed)
  • Helm (not listed)
  • Mac (not listed)
  • Windows (not listed)
  • Linux (not listed)
  • Any server (not listed)

Health score?A 0-100 score computed nightly from four factors: maintained (40%: recent commits and steady activity), popular (25%: GitHub stars, log scale), easy to install (20%: platforms, install notes, setup guides, Docker image) and light to run (15%: minimum RAM, ARM support). When the source code has been checked, a fifth factor, security check (20%: automated scan for committed secrets, unsafe desktop-app settings, known-vulnerable dependencies and project hygiene; any serious finding caps it at 20), is added. Factors without data are left out and the rest are rescaled.

28/100 Low

Maintained
0GitHub repository is archived
Popular
798,842 GitHub stars
Easy to install
351 install platform, install notes, Docker image
Light to run
108 GB minimum RAM

Availability: platform listings confirmed today, links checked 21 days ago

GitHub stars
8,842
Open issues
1485
Last commit
2026-01-28
Latest release
v2.2.0
License
Apache-2.0
Activity
stale

Checked 7 days ago - source: GitHub (intel/ipex-llm)

Resources & Compatibility

Minimum requirements: 8 GB RAM, 10 GB disk.

Checked 54 days ago - source: Docker Hub image tags and published docs

First-install notes

  • Install the Python agent dependencies Before running the PMT agent, install the Python packages lxml, requests, and urllib3 using pip (Python's package installer). Without them, the agent cannot collect or publish the telemetry data you want. source
  • Use the right XML config file Run the agent with sudo (admin privileges) and point the -s option to the pmt.xml configuration file that matches your platform. The wrong file or missing permissions can stop the agent from finding the right sensors. source
  • Open port 4317 for the collector The collector example starts a gRPC receiver on port 4317, where gRPC is a way for programs to call each other over the network. If other machines will send metrics there, keep port 4317 reachable or their data will not arrive. source
  • Enable ReBAR in BIOS Enable ReBAR (Resizable Base Address Register) in your BIOS/UEFI before first use; Intel GPU acceleration with IPEX-LLM requires it to be turned on. source
  • Archived upstream Intel has archived IPEX-LLM and will not provide maintenance, bug fixes, new releases, updates, or support; known security issues exist. source
  • Upstream status Intel has archived IPEX-LLM; it is unsupported, no longer updated, and has known security issues. source

Fetched 9 days ago - each note links to its source

Alternatives

No alternatives collected yet.

Common questions

What should I do if IPEX-LLM Ollama does not work on a PC with an AMD 8700F CPU?

One user reported that it works after reinstalling OneAPI and IPEX-LLM Ollama while following Intel's graphics installation guidance.

Can model behavior change when migrating from BigDL-LLM to IPEX-LLM?

Yes, a user found that a Vicuna 13B model stopped working after migrating, while a Phi-2-super model ran at about 7-8 tokens per second.

What trade-offs does consolidating inference into one Intel IPEX-LLM Ollama container involve?

It collapses two inference engines into a single container, which reduces moving parts and adds native FIM support, but drops the OpenVINO stack.

Answers sourced from community.intel.com, shadowcp.dev, www.reddit.com

Community

Collected 49 days ago - public community threads