OpenChat-Cuda
A self-hosted, offline, ChatGPT-like chatbot that runs open-source LLMs locally using CUDA on an NVIDIA GPU, keeping conversations private and local.
Platform availability
Available on 1 of 10 platforms
- Unraid
- TrueNAS (not listed)
- Umbrel (not listed)
- ZimaOS (not listed)
- Proxmox (not listed)
- Helm (not listed)
- Mac (not listed)
- Windows (not listed)
- Linux (not listed)
- Any server (not listed)
Health score?A 0-100 score computed nightly from four factors: maintained (40%: recent commits and steady activity), popular (25%: GitHub stars, log scale), easy to install (20%: platforms, install notes, setup guides, Docker image) and light to run (15%: minimum RAM, ARM support). When the source code has been checked, a fifth factor, security check (20%: automated scan for committed secrets, unsafe desktop-app settings, known-vulnerable dependencies and project hygiene; any serious finding caps it at 20), is added. Factors without data are left out and the rest are rescaled.
17/100 Low
- Maintained
- 0last commit 1170 days ago
- Popular
- 3767 GitHub stars
- Easy to install
- 351 install platform, install notes, Docker image
- Light to run
- 108 GB minimum RAM
- Security check
- 152 serious findings, 16 minor, CI, tests
Security check: 2 serious findings
- Uses form-data 4.0.0, which has a known critical vulnerability (npm)
- Uses i18next-fs-backend 2.1.1, which has a known critical vulnerability (npm)
Automated checks on commit 5781c9d, 12 Nov 2026. Not an audit.
Availability: platform listings confirmed today, links checked 19 days ago
- GitHub stars
- 67
- Open issues
- 12
- Last commit
- 2023-09-08
- Latest release
- v1.0.6
- License
- MIT
- Activity
- stale
Checked 6 days ago - source: GitHub (edgar971/open-chat)
Resources & Compatibility
Minimum requirements: 8 GB RAM.
Checked 54 days ago - source: Docker Hub image tags and published docs
First-install notes
- Set your API key first Create a .env.local settings file and add OPENAI_API_KEY (the key that lets the app talk to the AI) plus DEFAULT_MODEL (the model name to use by default). Without a valid key, chats will fail because the app has no permission to use the model. source
- Point UI to model server If you run a local OpenChat/CUDA backend, set OPENAI_API_HOST (where the app should send chat requests) to its web address and port, and use the key that backend expects. If this is wrong, the UI cannot reach the model and will error when you send a message. source
- Keep settings file safe Keep .env.local private because it can contain API keys that allow paid or authenticated model access. Anyone who gets it can use that access, so store it only where the app needs it. source
- NVIDIA GPU required This template only runs on an NVIDIA GPU. Install the NVIDIA driver plugin from the Unraid Community Applications store before starting the container, or the model server will fail to start. source
- NVIDIA GPU Install the Unraid NVIDIA-DRIVER plugin before starting OpenChat-Cuda. source
Fetched today - each note links to its source
Variants
Alternatives
No alternatives collected yet.
Common questions
Which compose command starts the 13B model?
Run docker compose -f docker-compose-13b.yml up -d. The 13B q4_0 model requires at least 16GB RAM.
Which local port should a reverse proxy target?
Use port 3000 because the README says the app is accessible at localhost:3000.
How do I set the model path and context length in Docker?
Add environment variables to the api service, such as MODEL=/path/to/your/model and N_CTX=4096. Additional API settings can also be passed as arguments to run.sh.
Can OpenChat 3.5 7B run on a Pascal GPU?
The source only shows a Reddit question asking whether it works. It does not include a confirmed result.
Answers sourced from github.com, www.reddit.com
Community
Collected 49 days ago - public community threads