llama.cpp

LLM inference in C/C++

2 views

A lightweight inference engine for running large language models on ordinary hardware, from laptops to servers, including on CPU alone. Includes a command line tool and an OpenAI-compatible server. MIT licensed.