https://blog.lmcache.ai/2025-03-06-benchmark/

[giscus[bot]]

https://blog.lmcache.ai/2025-03-06-benchmark/

A picture is worth a thousand words: Executive Summary: vLLM Production Stack, an open-source reference implementation of a cluster-wide, full-stack vLLM serving system, was first released in Jan 2025 by researchers from vLLM and UChicago. Since then, the system has gained popularity and attracted a growing open-source contributor community (check...

https://blog.lmcache.ai/2025-03-06-benchmark/

[panpan0000]

Hi, team, good benchmark report!
Just a question, do you have the benchmark for bigger output ( current is 9k input/10 output) ?
such a small output(10 tokens) may only cover the prefill stage performance, but I think the decode stage is more meaningful in real world and decode stage performance will be impacted by kv-cache offloading.