How AI is applied across API Evangelist and APIs.io. Read my AI disclosure →
API Evangelist API Evangelist
Discovery
Learnings
Guidance
Toolbox
Alignment
API Evangelist LLC

Pass@k vs Pass^k: Average Pass Rate Hides Flaky AI Agents

calendar_today September 29, 2026 person Samyak Goyal domain lambdatest

Pass@k asks whether any of k runs passed; pass^k asks whether all k did. See why average pass rate hides flaky AI agents and how many runs a pass^k gate needs.

open_in_new Read original post