A host-centric framework for scaling KV cache across host memory, filesystems, object stores, and remote peers — reducing recomputation and increasing serving capacity.
Tiered KV Cache Offloading in vLLM
calendar_today
September 10, 2026
person
Or Ozeri, Danny Harnik, Ronen Schaffer, Itay Etelis, Varun Sundar Rabindranath
domain
vllm