When Does RL Actually Help Fine-Tuning? A Difficulty-Controlled Study on Structured Generation

The uncomfortable question Reinforcement learning is often the finishing move of the modern fine-tuning stack: run SFT first, then add RL (GRPO, PPO, DPO) to squeeze out the last few points. In practice the return is wildly inconsistent — sometimes a real jump, sometimes nothing after a burned GPU budget. The folk rule “RL helps when the task is hard” is directionally right but too vague to budget against: it doesn’t say how much , which fields , or how to check in advance .