If you have wired an AI agent into a real production workflow, you have probably hit this wall; the agent is genuinely good at the task, but it is expensive to run it every single time , especially when a meaningful chunk of the requests it receives are things it has already solved before. That was exactly the situation I ran into. The setup looked like this;