Plan, divide, and conquer: How weak models excel at long context tasks

As context windows grow, LLM performance degrades in unexpected ways. We show how a “Divide & Conquer” framework — breaking long documents into parallel chunks with a planner, workers, and manager — lets smaller models like Llama-3-70B and Qwen-72B outperform GPT-4o single-shot.