LLM inference in C/C++ https://llama.app
  • C++ 55.6%
  • C 16.4%
  • Python 7.8%
  • Cuda 5%
  • TypeScript 4.5%
  • Other 10.5%
Find a file
Repository files (latest commit first)
Filename Latest commit message Latest commit date
2026-11-17 00:56:20 +08:00
.devops
.gemini
.github
.pi/gg
app
benches
ci
cmake
common
conversion
docs
examples
ggml MMVQ: Tune ne11 Gate for RDNA3_5 (#32308) 2026-11-17 00:56:20 +08:00
gguf-py
grammars
include
licenses
media
models
pocs
requirements
scripts
skills
src
tests
tools
ui/apps/chat
vendor
.clang-format
.clang-tidy
.dockerignore
.ecrc
.editorconfig
.flake8
.gitignore
.gitmodules
.pre-commit-config.yaml
AGENTS.md
AUTHORS
build-xcframework.sh
CLAUDE.md
CMakeLists.txt
CMakePresets.json
CODEOWNERS
CONTRIBUTING.md
convert_hf_to_gguf.py
convert_hf_to_gguf_update.py
convert_llama_ggml_to_gguf.py
convert_lora_to_gguf.py
flake.nix
LICENSE
Makefile
mypy.ini
pyproject.toml
pyrightconfig.json
README.md
requirements.txt
SECURITY.md
ty.toml

llama.cpp

llama.cpp

Description

The main goal of llama.cpp is to enable LLM (and MLLM) inference with minimal setup and state-of-the-art performance on a wide range of hardware - locally and in the cloud.

The llama.cpp project is built on top of the ggml library.

Quick start

A few options to get llama.cpp installed on your machine:

# curl
curl -LsSf https://llama.app/install.sh | sh

# powershell
irm https://llama.app/install.ps1 | iex

Once installed:

# Download and run a model directly from Hugging Face
llama cli -hf ggml-org/Qwen3.5-0.8B-GGUF

# Launch OpenAI-compatible API server
llama serve -hf ggml-org/Qwen3.5-0.8B-GGUF
VLM session with `llama cli` VLM session with llama cli Llama Chat, the built-in chat app, against `llama serve` running Qwen 3.6 Llama Chat, the built-in chat app, against llama serve

Documentation

Tools

Development

Contributing

  • Contributors can open PRs
  • Collaborators will be invited based on contributions
  • Maintainers can push to branches in the llama.cpp repo and merge PRs into the master branch
  • Any help with managing issues, PRs and projects is very appreciated!
  • Read the CONTRIBUTING.md for more information

Acknowledgements

  • yhirose/cpp-httplib - Single-header HTTP server, used by llama-server - MIT license
  • nothings/stb - Single-header image format decoder, used by multimodal subsystem - Public domain
  • nlohmann/json - Single-header JSON library, used by various tools/examples - MIT License
  • mackron/miniaudio - Single-header audio format decoder, used by multimodal subsystem - Public domain
  • sheredom/subprocess.h - Single-header process launching solution for C and C++ - Public domain