Serge

Serge is an archived self-hosted chat interface that runs local large language models (LLMs) with llama.cpp, letting users chat privately without API keys,…

  • AI
Serge is an archived self-hosted chat interface that runs local large language models (LLMs) with llama.cpp, letting users chat privately without API keys, including on CPU-only systems.

Platform availability

Available on 1 of 10 platforms

  • Unraid
  • TrueNAS (not listed)
  • Umbrel (not listed)
  • ZimaOS (not listed)
  • Proxmox (not listed)
  • Helm (not listed)
  • Mac (not listed)
  • Windows (not listed)
  • Linux (not listed)
  • Any server (not listed)

Health score?A 0-100 score computed nightly from four factors: maintained (40%: recent commits and steady activity), popular (25%: GitHub stars, log scale), easy to install (20%: platforms, install notes, setup guides, Docker image) and light to run (15%: minimum RAM, ARM support). When the source code has been checked, a fifth factor, security check (20%: automated scan for committed secrets, unsafe desktop-app settings, known-vulnerable dependencies and project hygiene; any serious finding caps it at 20), is added. Factors without data are left out and the rest are rescaled.

40/100 Fair

Maintained
0GitHub repository is archived
Popular
755,692 GitHub stars
Easy to install
351 install platform, install notes, Docker image
Light to run
304 GB minimum RAM
Security check
88no serious findings, 2 minor, CI, tests

Availability: platform listings confirmed today

GitHub stars
5,692
Open issues
33
Last commit
2025-06-22
Latest release
0.9.0
License
Apache-2.0
Activity
stale

Checked 6 days ago - source: GitHub (serge-chat/serge)

Resources & Compatibility

Minimum requirements: 4 GB RAM.

Checked 54 days ago - source: Docker Hub image tags and published docs

First-install notes

  • Create the project with the Sanity CLI Run the terminal command npm create sanity@latest -- --template serge-0v/next-js-sanity-starter once. It sets up the Sanity content project, asks for the connection details and CORS settings, and gets the starter ready so the site can find its content. source
  • Fill in .env.local before starting Put your Sanity project ID, dataset (the content collection the site reads), read token, and site URL in the .env.local file before starting. These connect the website to your content and preview system, and without them pages may load blank or fail to work. source
  • Add sample content if pages are empty Import the included sample data into the production dataset (the main content collection) using the documented dataset import command. This gives you starter pages to test with, because a fresh project will have no content until you add it. source
  • Memory requirements llama.cpp will crash if there is not enough free RAM for the selected model; a 7B model needs about 4.5GB, 13B about 12GB, and 30B about 20GB of free RAM. source
  • Docker Use the Docker quick-start, persist the model weights directory, and open the web UI to download a model. source
  • Docker quick start Run the container and mount a persistent volume for model weights so downloaded models persist across restarts. source

Fetched today - each note links to its source

Alternatives (3)

  • ChatGPT - ChatGPT is a generative artificial intelligence chatbot developed by OpenAI and launched in 2022.
  • GPT4ALL - GPT4All runs open-source large language models (LLMs) privately on your own hardware, providing a local chat interface and server that need…
  • llamagpt - LlamaGPT is a self-hosted, offline, private ChatGPT-like chatbot for Umbrel that runs a Llama 2 model locally so no data leaves your device…

Checked 40 days ago - source: AlternativeTo

Common questions

Which port should I expose for Serge?

The Docker examples map port 8008. The web interface and API are served from that port.

Which Docker volumes should I mount to preserve models and app data?

Mount weights to /usr/src/app/weights for model files. Mount datadb to /data/db for app data, with the default database file at /data/db/sql_app.db.

Which environment variables should I set for authentication?

Set SERGE_JWT_SECRET to a random string for auth token encryption. SERGE_SESSION_EXPIRY controls reauthentication and defaults to 60 minutes.

Where can I find the API documentation?

The API documentation is available under the /api/docs path.

Answers sourced from github.com

Community

Collected 48 days ago - public community threads