A push-based NIXL connector for vLLM disaggregated serving: the prefill instance writes KV directly into decode memory over RDMA, taking the router relay, decode-side allocation, and transfer setup off the TTFT critical path.
Push-Based KV Transfer to Improve TTFT and TPOT for Disaggregated Inference in vLLM
calendar_today
October 14, 2026
person
Sunita Nadampalli (AWS), Nicolò Lucchesi (Mistral)
domain
vllm