Speech-To-Speech-CPU

Speech-To-Speech-CPU runs Hugging Face's open-source realtime voice-agent pipeline (Apache-2.0, 13k+ stars): voice activity detection, speech-to-text…

  • AI
Speech-To-Speech-CPU runs Hugging Face's open-source realtime voice-agent pipeline (Apache-2.0, 13k+ stars): voice activity detection, speech-to-text (faster-whisper), any OpenAI-compatible LLM as the brain, and Kokoro text-to-speech — served as one OpenAI Realtime WebSocket server. Point any Realtime-API client at it instead of OpenAI's cloud: Conduit's "local listening" (speech server), OpenAI Agents SDK clients, LiveKit, or its own browser demo. This template runs a CPU image built and published by Aurial (upstream ships CUDA-only and no registry image), so it needs no GPU and can run alongside GPU inference servers. It has no web page of its own: it is a backend that apps connect to. Set its three LLM fields to point at an existing OpenAI-compatible server (Strata, Radiance, Ollama, LM Studio, …); the key is stored masked, never on the command line. One voice session at a time.

Platform availability

Available on 1 of 10 platforms

  • Unraid
  • TrueNAS (not listed)
  • Umbrel (not listed)
  • ZimaOS (not listed)
  • Proxmox (not listed)
  • Helm (not listed)
  • Mac (not listed)
  • Windows (not listed)
  • Linux (not listed)
  • Any server (not listed)

Health score?A 0-100 score computed nightly from four factors: maintained (40%: recent commits and steady activity), popular (25%: GitHub stars, log scale), easy to install (20%: platforms, install notes, setup guides, Docker image) and light to run (15%: minimum RAM, ARM support). When the source code has been checked, a fifth factor, security check (20%: automated scan for committed secrets, unsafe desktop-app settings, known-vulnerable dependencies and project hygiene; any serious finding caps it at 20), is added. Factors without data are left out and the rest are rescaled.

82/100 Excellent

Maintained
100last commit 10 days ago
Popular
8313,679 GitHub stars
Easy to install
351 install platform, install notes, Docker image
Light to run
No data
Security check
94no serious findings, 1 minor, CI, tests

Based on 4 of 5 factors.

Availability: platform listings confirmed today

GitHub stars
13,679
Open issues
69
Last commit
2026-11-11
Latest release
v1.1.0
License
Apache-2.0
Activity
declining

Checked 10 days ago - source: GitHub (huggingface/speech-to-speech)

Resources & Compatibility

Requirements not published yet.

Checked 10 days ago - source: Docker Hub image tags and published docs

First-install notes

  • Fix audio tool install permission errors Run the setup cell as root (the admin user) or install sox, libsndfile1, and ffmpeg in advance. If the notebook runtime says apt-get is not root, those audio tools won't install and speech processing can break. source
  • Restart the runtime after setup After the setup cell finishes, restart the notebook runtime (the temporary environment that runs the code). This makes the newly installed NeMo speech toolkit and Python packages take effect; without it, later cells may use old versions and fail. source
  • Start the client in another terminal Open another terminal (a command-line window), then in the client folder use npm (Node's package manager) to run npm install and npm run dev. This brings up the local web interface so you can interact with the app instead of only running the backend. source
  • Runs entirely on your machine Starts with a single command and needs no API key or cloud service. It exposes an OpenAI Realtime WebSocket/WebRTC endpoint, so Realtime-API clients (Conduit, LiveKit, the OpenAI Agents SDK, or its own browser demo) can connect to it directly. source
  • Setup Point the three LLM fields at an existing OpenAI-compatible LLM server (e.g. Ollama or LM Studio); the key is stored masked and never appears on the command line. Expect one concurrent voice session. source

Fetched 5 days ago - each note links to its source

Alternatives

No alternatives collected yet.

Common questions

Do I need an OpenAI or other cloud API key to run it locally?

No. It is described as starting a conversation with one command without an API key or cloud service.

What should I do when using speakers so the assistant does not hear itself?

Add --echo-cancellation when running on speakers. This lets you still interrupt by voice; headphones do not need it.

Does the first run take extra time?

Yes, the first run downloads the models. Allow a few minutes before it is ready to speak.

Can existing OpenAI Realtime clients connect to it?

Yes. The server speaks the OpenAI Realtime protocol over WebSocket and WebRTC, so Realtime API clients can point at it.

How can I enable more verbose or diagnostic output?

Add --verbose for diagnostic output, and use --log_level debug for more detailed local logs. Raw event output is available with local --local_audio_print_json or talk --print-json.

Is the default Linux Qwen3-TTS wheel generic across systems?

No. The default Linux wheel targets CUDA 12.8 and Ubuntu 24.04, and the README points to the CUDA note for other systems.

Answers sourced from github.com

Community

No community discussions collected yet.