llama.cpp: LLM and VLM inference in C/C++

Key points

1. It ships a command line interface, an OpenAI-compatible API server, and a web chat UI, installable as an app, via Docker, from pre-built binaries, or from source.
2. Backends cover Apple Silicon with ARM NEON, Accelerate and Metal, x86 with AVX through AVX512 and AMX, NVIDIA CUDA, AMD HIP, Vulkan and SYCL, plus hybrid CPU and GPU inference.
3. The repository showed 124.9k stars and 22.0k forks with 778 open issues and roughly 1.4k open pull requests, and quantization spans 1.5-bit to 8-bit so output quality depends on the level chosen.