The integration of Large Language Models (LLMs) into agentic architectures has substantially increased token consumption, raising concerns about their energy footprint. Yet, estimating the energy consumption of proprietary API-based models remains challenging, and existing approaches focus primarily on generated output tokens while overlooking the impact of the large input contexts characteristic of agentic workflows. To address this gap, we systematically profile compute time and energy consumption across a wide range of prefill (input) and decode (output) token lengths.
The Real Cost of Context: Predicting Compute Time and Energy Consumption in LLM Prefill and Decode Phases
calendar_today
October 30, 2026
domain
machinelibrary-ai