Launch benchmarks from April 2025: the NVIDIA RTX 5090 served Qwen2-0.5B at 65,000+ tokens/sec and 250+ requests/sec with vLLM at 1024 concurrent prompts.
The RTX 5090 Is Here: Serve 65,000+ Tokens Per Second on Runpod
calendar_today
October 16, 2026
domain
runpod