The physical layer is the new compiler

Scaling LLM inference to the edge has mostly been a game of squeezing weights into local SRAM. The industry has focused on quantization, pruning, and memory management. We treat the device as an isolated island of compute, and if we want to scale, we try to build bigger islands.