AIGatewayBench measures the overhead an AI gateway adds on top of the upstream model, isolated against a deterministic mock, across LiteLLM (Rust), LiteLLM (Python v1), Portkey, and Bifrost. The LiteLLM Rust gateway has the lowest overhead and memory footprint of the four.
Benchmarking the LiteLLM Rust AI Gateway: Overhead, Memory, and Cost
calendar_today
July 22, 2026
domain
litellm