Android Bench 2.0: Pushing the frontier with challenging long-horizon tasks

Posted by Matthew McCullough, VP, Product Management, Android Developer When we first launched Android Bench, we built a rigorous foundation for evaluating how large language models (LLMs) assist developers with real-world Android tasks. As AI models and agents rapidly evolve, we’ve been updating our methodology, such as aligning our benchmark framework with the Harbor framework . Today we’re releasing the first set of long-horizon tasks (LHT) , which are tasks of great complexity that take an engineer multiple days or even a week to complete.