vllm-metal brings vLLM’s paged, continuously batched serving stack to Apple Silicon, with flatter TTFT under concurrent agent load, batched MTP, and automatic M5 prefill acceleration.
Announcing vllm-metal: Concurrent Serving on Apple Silicon
calendar_today
September 22, 2026
person
Ranran Haoran Zhang, Lik Xun Yuan, Chao Ju Chen, Eric Curtin, Michael Goin
domain
vllm