How AI is applied across API Evangelist and APIs.io. Read my AI disclosure →
API Evangelist API Evangelist
Discovery
Learnings
Guidance
Toolbox
Alignment
API Evangelist LLC

Does Context Length Affect Inference Cost Linearly? We Measured Why It Doesn't

calendar_today September 1, 2026 person Vinayak Baranwal domain digital-ocean

Long-context pricing is billed as linear, but serving cost is not. We measured Ministral 3 14B on a DigitalOcean H200 across 2K-256K tokens: KV cache pool capacity drives a 3.84x cost-per-token rise and a 256K crossover against the $0.20/1M serverless rate.

open_in_new Read original post