How AI is applied across API Evangelist and APIs.io. Read my AI disclosure →
API Evangelist API Evangelist
Discovery
Learnings
Guidance
Toolbox
Alignment
API Evangelist LLC

Benchmarking tiered KV cache offloading on H100 GPUs in OpenShift AI 3.5

calendar_today November 2, 2026 person Ramesh Doddaiah, Alberto Perdomo, Nathan Scott, Ashish Kamra, Michey Mehta, Yuchen Fama, Naina Singh domain red-hat

Long-context workloads are quickly becoming the norm for enterprise AI. Agentic applications carry an ever-growing history of tool calls and intermediate results, retrieval-augmented generation prepends thousands of tokens of retrieved context, and multi-turn assistants resend the whole conversation with every turn. In all of these cases, the model sees large parts of the same prompt again and again.

open_in_new Read original post