TL;DR A model can summarize a long document in a test request and still miss your latency target once several requests arrive together. For an agent, a serving route’s output cap or chat template may rule it out before you run a load test. The Mistral vs Llama vs Mixtral decision starts with the workload.