# NVIDIA NIM Single

NVIDIA NIM Single is a self-hosted ai app. NVIDIA NIM (Inference Microservices) is a containerized inference server that runs local large language models on NVIDIA GPUs with CUDA acceleration, exposing an OpenAI-compatible API for developers and homelabbers who... It is open source and self-hosted, installable on Unraid.

Canonical page: https://ourlocalhost.com/app/3127/nvidia-nim-single

## Description

NVIDIA NIM (Inference Microservices) is a containerized inference server that runs local large language models on NVIDIA GPUs with CUDA acceleration, exposing an OpenAI-compatible API for developers and homelabbers who want self-hosted LLM access without cloud dependencies.

## Platform availability

Available on 1 of 10 platforms: Unraid.

## Health score

48/100 (Fair), based on 3 of 4 factors.
- Maintained: 79 (last commit 30 days ago)
- Popular: 10 (2 GitHub stars)
- Easy to install: 35 (1 install platform, install notes, Docker image)

## Links

- GitHub: https://github.com/PikkonMG/unraid-docker-templates
- Official website: https://build.nvidia.com/
- Unraid: https://ca.unraid.net/apps/nvidia-nim-single-1on68tb1e3l0er?q=nvidia-nim-single
- Documentation: https://docs.api.nvidia.com/

## Common questions

### How do I switch the model in an NVIDIA NIM LLM container?

Update both the Docker image tag and the NIM_MODEL_NAME variable to matching values. The default model is meta/llama-3.2-3b-instruct.

### Why does a client call to a NIM endpoint fail authentication?

Set invoke_url to your NIM endpoint and add your API key to the Authorization header. Without these, the service cannot identify you and rejects the request.

### How can I reduce memory usage for a NIM running in WSL2?

Set autoMemoryReclaim=dropcache in the WSL2 configuration. These commands require sudo access.

---
Source: Our Local Host (https://ourlocalhost.com/app/3127/nvidia-nim-single), the self-hosted and open-source app directory.
