What is GPU Memory and Why it Matters for LLM Inference

This guide explores how GPU memory (VRAM) functions during LLM inference and why models that appear to fit on paper often experience out-of-memory errors. It covers memory allocation across model weights, KV cache expansion, and practical optimization strategies including quantization and distributed inference techniques.