I’ve been experimenting with speculative decoding for LLM inference on my RTX 5070 (12GB VRAM). Specifically, I implemented Medusa-style multi-token prediction heads — where a small auxiliary head predicts the next 4-5 tokens in parallel, and the main model verifies them. Results so far: - ~2x throughput improvement when verification succeeds (draft accepted) - Draft generation is fast (~0.1ms per token on CPU) - Verification step becomes the bottleneck — can’t skip it even for simple queries -