The CPU is back: Rethinking the CPU-GPU split for LLM inference

For the past 3 years, graphics processing units (GPUs) have dominated the large language model (LLM) conversation. In traditional chatbot applications, central processing units (CPUs) provide a fraction of the total compute per request, while GPUs do the heavy lifting. However, inference isn’t a single model answering a single question.