Granite 3 2B Instruct
Granite 3 2B Instruct is a compact open-source instruction-tuned language model from IBM that runs privately on your own hardware for local chat, drafting,…
Platform availability
Available on 0 of 10 platforms
- Unraid (not listed)
- TrueNAS (not listed)
- Umbrel (not listed)
- ZimaOS (not listed)
- Proxmox (not listed)
- Helm (not listed)
- Mac (not listed)
- Windows (not listed)
- Linux (not listed)
- Any server (not listed)
Health score?A 0-100 score computed nightly from four factors: maintained (40%: recent commits and steady activity), popular (25%: GitHub stars, log scale), easy to install (20%: platforms, install notes, setup guides, Docker image) and light to run (15%: minimum RAM, ARM support). When the source code has been checked, a fifth factor, security check (20%: automated scan for committed secrets, unsafe desktop-app settings, known-vulnerable dependencies and project hygiene; any serious finding caps it at 20), is added. Factors without data are left out and the rest are rescaled.
35/100 Low
- Maintained
- 0GitHub repository is archived
- Popular
- 621,249 GitHub stars
- Easy to install
- 250 install platforms, install notes, Docker image
- Light to run
- 403.66016 GB minimum RAM, runs on ARM
- Security check
- 75no serious findings
- GitHub stars
- 1,249
- Open issues
- 5
- Last commit
- 2024-11-13
- License
- Apache-2.0
- Activity
- stale
Checked 16 days ago - source: GitHub (ibm-granite/granite-code-models)
Resources & Compatibility
Minimum requirements: 3.66016 GB RAM. ARM (e.g. Raspberry Pi) is supported.
Checked 54 days ago - source: Docker Hub image tags and published docs
First-install notes
- Install the model's Python packages Before starting, use pip (Python's package installer) to add PyTorch, vLLM (a model-serving tool) 0.6.6, and transformers 4.49 or newer. These are the main tools that let the model load, so a missing or old package usually stops startup. source
- Add accelerate if you see a missing package error If the model startup complains that accelerate is missing, use pip to install accelerate. This small helper package is needed for some initialization steps and prevents the setup from failing. source
- Know the SGLang port before opening your network The example SGLang (a model-serving program) command uses port 30000 and --host 0.0.0.0, meaning it can be reached from other devices on your network. Open port 30000 only if another device should connect; if you only use it locally, you don't need to expose it. source
Fetched 53 days ago - each note links to its source
Variants
Alternatives (4)
- Civitai - Civitai is the only Model-sharing hub for the AI art generation community!
- Cleverbot - Cleverbot is an AI web application that learns how to mimic human conversations by conversing with humans.
- Kuki - Browser-based AI chat bot.
- Replicate Codex - Search Replicate AI models.
Checked 2 days ago - source: AlternativeTo
Common questions
Can chain-of-thought reasoning be enabled or disabled for Granite 3.2 2B Instruct?
Yes. The IBM community post says the 8B and 2B Instruct models offer experimental chain-of-thought reasoning that can be toggled on or off to help follow complex instructions.
Which port does the example SGLang command use, and should it be opened on the network?
The example SGLang command uses port 30000 and --host 0.0.0.0. Open port 30000 only if another device should connect; if you only use it locally, you don't need to expose it.
What should be done if model startup complains that accelerate is missing?
Use pip to install accelerate. The discussion says this small helper package is needed for some initialization steps and prevents the setup from failing.
Is there a Raspberry Pi 5 guide for running Granite 3.2-2B locally?
Yes. The granitepi repository contains a diagram and a walkthrough guide for a DIY Raspberry Pi 5 + IBM Granite 3.2-2b local model.
Answers sourced from community.ibm.com, github.com, huggingface.co, huggingface.co
Community
- New Coding Model from IBM (IBM Granite) : r/LocalLLaMA - Reddit (reddit)
- IBM Granite 3.2: Reasoning, vision, forecasting and more (other)
Collected 48 days ago - public community threads