Granite 3.1 1B A400M Instruct
Granite 3.1 1B A400M Instruct is a lightweight, multilingual IBM language model that runs on your own hardware, letting homelabbers privately answer…
Platform availability
Available on 0 of 10 platforms
- Unraid (not listed)
- TrueNAS (not listed)
- Umbrel (not listed)
- ZimaOS (not listed)
- Proxmox (not listed)
- Helm (not listed)
- Mac (not listed)
- Windows (not listed)
- Linux (not listed)
- Any server (not listed)
Health score?A 0-100 score computed nightly from four factors: maintained (40%: recent commits and steady activity), popular (25%: GitHub stars, log scale), easy to install (20%: platforms, install notes, setup guides, Docker image) and light to run (15%: minimum RAM, ARM support). When the source code has been checked, a fifth factor, security check (20%: automated scan for committed secrets, unsafe desktop-app settings, known-vulnerable dependencies and project hygiene; any serious finding caps it at 20), is added. Factors without data are left out and the rest are rescaled.
36/100 Low
- Maintained
- 0GitHub repository is archived
- Popular
- 43147 GitHub stars
- Easy to install
- 250 install platforms, install notes, Docker image
- Light to run
- 801 GB minimum RAM, runs on ARM
- Security check
- 75no serious findings
- GitHub stars
- 147
- Open issues
- 5
- Last commit
- 2025-06-25
- License
- Apache-2.0
- Activity
- stale
Checked 3 days ago - source: GitHub (ibm-granite/granite-3.1-language-models)
Resources & Compatibility
Minimum requirements: 1 GB RAM. ARM (e.g. Raspberry Pi) is supported.
Checked 54 days ago - source: Docker Hub image tags and published docs
First-install notes
- Install the model’s required Python tools Use pip (Python’s package installer) to install PyTorch (the math toolkit for AI models) and vLLM 0.6.6 (the program that runs the model). If these are missing or the wrong version, the model won’t start. source
- Update the Hugging Face helper library Install transformers version 4.49 or newer, which helps the app read the model’s instructions and image inputs. An older version can make setup fail. source
- Add the missing helper package If you get a missing-dependency error, install the accelerate package with pip. It helps the model load and can stop that error from blocking you. source
- Obtain model weights Download the model weights from the Hugging Face collection linked in the repository. source
- Hugging Face weights Download the model weights from the Hugging Face collection linked in the repository. source
- Archived upstream repository The upstream GitHub repository is archived; use the Docker Hub image or Hugging Face weights rather than expecting new repository updates. source
Fetched 2 days ago - each note links to its source
Variants
Alternatives
No alternatives collected yet.
Common questions
What context length should I expect when running Granite 3.1?
Granite 3.1 extends the context length of Granite 3.0 from 4K to 128K.
How many experts does the 1B-A400M MoE use at inference?
It is a Mixture-of-Experts decoder transformer with 32 experts and 8 active.
Can I offer it as a commercial self-hosted API?
Yes. The models are publicly released under an Apache 2.0 license for both research and commercial use.
Does the instruct model support tool calling workflows?
Granite 3.1 instruction models provide an improved developer experience for function-calling and RAG generation tasks.
What data does the 1B-A400M instruct checkpoint use for tuning?
It is finetuned from the base model using open source instruction datasets with permissive licenses and internally collected synthetic datasets tailored for long context.
Answers sourced from github.com, huggingface.co, slm.expert
Community
Collected 48 days ago - public community threads