LLaMA Factory(Nvidia GPU)

LLaMA Factory is an open-source, self-hosted framework for fine-tuning and serving large language models on NVIDIA GPU hardware.

  • AI
LLaMA Factory is an open-source, self-hosted framework for fine-tuning and serving large language models on NVIDIA GPU hardware. It provides a web interface for more than 100 models and supports LoRA, QLoRA, full-parameter and freeze training, quantization, multi-GPU training, and an integrated inference API for self-hosters who want a unified model-customization workflow.

Platform availability

Available on 1 of 10 platforms

  • Unraid (not listed)
  • TrueNAS (not listed)
  • Umbrel (not listed)
  • ZimaOS
  • Proxmox (not listed)
  • Helm (not listed)
  • Mac (not listed)
  • Windows (not listed)
  • Linux (not listed)
  • Any server (not listed)

Health score?A 0-100 score computed nightly from four factors: maintained (40%: recent commits and steady activity), popular (25%: GitHub stars, log scale), easy to install (20%: platforms, install notes, setup guides, Docker image) and light to run (15%: minimum RAM, ARM support). When the source code has been checked, a fifth factor, security check (20%: automated scan for committed secrets, unsafe desktop-app settings, known-vulnerable dependencies and project hygiene; any serious finding caps it at 20), is added. Factors without data are left out and the rest are rescaled.

85/100 Excellent

Maintained
100last commit 15 days ago
Popular
9875,942 GitHub stars
Easy to install
351 install platform, install notes, Docker image
Light to run
No data
Security check
89no serious findings, 1 minor, CI, tests

Based on 4 of 5 factors.

Availability: platform listings confirmed today

GitHub stars
75,942
Open issues
1222
Last commit
2026-11-06
Latest release
v0.9.5
License
Apache-2.0
Activity
steady

Checked 13 days ago - source: GitHub (hiyouga/LLaMA-Factory)

Resources & Compatibility

ARM (e.g. Raspberry Pi) is supported.

Checked 54 days ago - source: Docker Hub image tags and published docs

First-install notes

  • Set API port and GPU before launching Before running the API example, set API_PORT=8000 (or another free port) and CUDA_VISIBLE_DEVICES (the GPU number shown to the program) to the card you want to use, and open that port if another machine will call it. If the port or GPU is wrong, the service can fail to start, use the wrong card, or your client cannot connect. source
  • Fill in the SFT config before training Use the sample SFT config as your starting checklist, and change model_name_or_path, dataset, and output_dir to match what you plan to train and where you want the results saved. If the model/dataset paths are wrong or the batch size is too large, training can crash, run out of memory, or save the wrong files. source
  • Use WebUI for first-time fine-tuning For a first-time run, launch the WebUI with `llamafactory-cli webui` and use its forms to pick the model, dataset, and LoRA settings. That avoids hand-editing YAML and helps you see which settings need to be filled in before training. source

Fetched 53 days ago - each note links to its source

Alternatives (1)

  • Unsloth - Unsloth is a cross-platform desktop app (Windows, macOS, Linux) that lets you download, run, and fine-tune open-source large language model…

Checked 39 days ago - source: AlternativeTo

Common questions

Can installing LLaMA Factory cause a PyTorch version conflict?

Yes. On a DGX Spark, the setup playbook can hit a conflict between the Docker environment and the LLaMA installation at step 4.

What should I set before starting the LLaMA Factory API server?

Set API_PORT to a free port such as 8000 and CUDA_VISIBLE_DEVICES to the GPU you want to use. Open that port if another machine will call it.

Which SFT config values must I adjust before training?

Use the sample SFT config as a starting checklist and change model_name_or_path, dataset, and output_dir. Wrong paths or a batch size that is too large can crash training, exhaust memory, or save the wrong files.

Can I launch a first training run without editing YAML files?

Yes. Launch the WebUI with llamafactory-cli webui and use its forms to choose the model, dataset, and LoRA settings.

Answers sourced from forums.developer.nvidia.com, llamafactory.readthedocs.io, llamafactory.readthedocs.io, llamafactory.readthedocs.io

Community

Collected 48 days ago - public community threads