llm-serve
A from-scratch LLM inference server, measured honestly on a CPU: what continuous batching and a paged KV cache really buy,
how the scheduler and memory manager are built, how correctness is proven, how it is operated, and what broke along the way.
This report is interactive (charts, diagrams and animations), so it needs JavaScript. Every number in it is generated from the raw benchmark data committed in the repository.