← Directory

Granite 3.1 1B A400M Instruct

Granite 3.1 1B A400M Instruct is a lightweight, multilingual IBM language model that runs on your own hardware, letting homelabbers privately answer…

  • AI
Granite 3.1 1B A400M Instruct is a lightweight, multilingual IBM language model that runs on your own hardware, letting homelabbers privately answer questions, write and summarise text, and perform coding, reasoning, and tool-use tasks.

Platform availability

Available on 0 of 6 install platforms

  • Unraid (not listed)
  • TrueNAS (not listed)
  • Umbrel (not listed)
  • ZimaOS (not listed)
  • Proxmox (not listed)
  • Helm (not listed)

Also on Docker Hub and GitHub

Health score

28/100 Low

Maintained
0GitHub repository is archived
Popular
43147 GitHub stars
Easy to install
250 install platforms, install notes, Docker image
Light to run
801 GB minimum RAM, runs on ARM
How is this calculated?

A 0-100 score computed nightly from four factors: maintained (40%: recent commits and steady activity), popular (25%: GitHub stars, log scale), easy to install (20%: platforms, install notes, setup guides, Docker image) and light to run (15%: minimum RAM, ARM support). Factors without data are left out and the rest are rescaled.

GitHub stars
147
Open issues
5
Last commit
2025-06-25
License
Apache-2.0
Activity
stale

Checked 7 days ago - source: GitHub (ibm-granite/granite-3.1-language-models)

Resources & Compatibility

Minimum requirements: 1 GB RAM. ARM (e.g. Raspberry Pi) is supported.

Checked 7 days ago - source: Docker Hub image tags and published docs

First-install notes

  • Install the model’s required Python tools Use pip (Python’s package installer) to install PyTorch (the math toolkit for AI models) and vLLM 0.6.6 (the program that runs the model). If these are missing or the wrong version, the model won’t start. source
  • Update the Hugging Face helper library Install transformers version 4.49 or newer, which helps the app read the model’s instructions and image inputs. An older version can make setup fail. source
  • Add the missing helper package If you get a missing-dependency error, install the accelerate package with pip. It helps the model load and can stop that error from blocking you. source

Fetched 7 days ago - each note links to its source

Alternatives

No alternatives collected yet.

Common questions

What context length should I expect when running Granite 3.1?

Granite 3.1 extends the context length of Granite 3.0 from 4K to 128K.

How many experts does the 1B-A400M MoE use at inference?

It is a Mixture-of-Experts decoder transformer with 32 experts and 8 active.

Can I offer it as a commercial self-hosted API?

Yes. The models are publicly released under an Apache 2.0 license for both research and commercial use.

Does the instruct model support tool calling workflows?

Granite 3.1 instruction models provide an improved developer experience for function-calling and RAG generation tasks.

What data does the 1B-A400M instruct checkpoint use for tuning?

It is finetuned from the base model using open source instruction datasets with permissive licenses and internally collected synthetic datasets tailored for long context.

Answers sourced from github.com, huggingface.co, slm.expert

Community

Collected 2 days ago - public community threads