LLaMA Factory(Nvidia GPU)
LLaMA Factory is an open-source, self-hosted framework for fine-tuning and serving large language models on NVIDIA GPU hardware.
Platform availability
Available on 1 of 10 platforms
- Unraid (not listed)
- TrueNAS (not listed)
- Umbrel (not listed)
- ZimaOS
- Proxmox (not listed)
- Helm (not listed)
- Mac (not listed)
- Windows (not listed)
- Linux (not listed)
- Any server (not listed)
Health score?A 0-100 score computed nightly from four factors: maintained (40%: recent commits and steady activity), popular (25%: GitHub stars, log scale), easy to install (20%: platforms, install notes, setup guides, Docker image) and light to run (15%: minimum RAM, ARM support). When the source code has been checked, a fifth factor, security check (20%: automated scan for committed secrets, unsafe desktop-app settings, known-vulnerable dependencies and project hygiene; any serious finding caps it at 20), is added. Factors without data are left out and the rest are rescaled.
85/100 Excellent
- Maintained
- 100last commit 15 days ago
- Popular
- 9875,942 GitHub stars
- Easy to install
- 351 install platform, install notes, Docker image
- Light to run
- No data
- Security check
- 89no serious findings, 1 minor, CI, tests
Based on 4 of 5 factors.
Availability: platform listings confirmed today
- GitHub stars
- 75,942
- Open issues
- 1222
- Last commit
- 2026-11-06
- Latest release
- v0.9.5
- License
- Apache-2.0
- Activity
- steady
Checked 13 days ago - source: GitHub (hiyouga/LLaMA-Factory)
Resources & Compatibility
ARM (e.g. Raspberry Pi) is supported.
Checked 54 days ago - source: Docker Hub image tags and published docs
First-install notes
- Set API port and GPU before launching Before running the API example, set API_PORT=8000 (or another free port) and CUDA_VISIBLE_DEVICES (the GPU number shown to the program) to the card you want to use, and open that port if another machine will call it. If the port or GPU is wrong, the service can fail to start, use the wrong card, or your client cannot connect. source
- Fill in the SFT config before training Use the sample SFT config as your starting checklist, and change model_name_or_path, dataset, and output_dir to match what you plan to train and where you want the results saved. If the model/dataset paths are wrong or the batch size is too large, training can crash, run out of memory, or save the wrong files. source
- Use WebUI for first-time fine-tuning For a first-time run, launch the WebUI with `llamafactory-cli webui` and use its forms to pick the model, dataset, and LoRA settings. That avoids hand-editing YAML and helps you see which settings need to be filled in before training. source
Fetched 53 days ago - each note links to its source
Variants
Alternatives (1)
- Unsloth - Unsloth is a cross-platform desktop app (Windows, macOS, Linux) that lets you download, run, and fine-tune open-source large language model…
Checked 39 days ago - source: AlternativeTo
Common questions
Can installing LLaMA Factory cause a PyTorch version conflict?
Yes. On a DGX Spark, the setup playbook can hit a conflict between the Docker environment and the LLaMA installation at step 4.
What should I set before starting the LLaMA Factory API server?
Set API_PORT to a free port such as 8000 and CUDA_VISIBLE_DEVICES to the GPU you want to use. Open that port if another machine will call it.
Which SFT config values must I adjust before training?
Use the sample SFT config as a starting checklist and change model_name_or_path, dataset, and output_dir. Wrong paths or a batch size that is too large can crash training, exhaust memory, or save the wrong files.
Can I launch a first training run without editing YAML files?
Yes. Launch the WebUI with llamafactory-cli webui and use its forms to choose the model, dataset, and LoRA settings.
Answers sourced from forums.developer.nvidia.com, llamafactory.readthedocs.io, llamafactory.readthedocs.io, llamafactory.readthedocs.io
Community
- Building a machine for self-hosted LLaMA - will 2 x RTX 3090 ... - Reddit (reddit)
- LLama factory conflicts torch version - NVIDIA Developer Forums (forum)
Collected 48 days ago - public community threads