Sparser, Faster, Lighter Transformer Language Models

–> How do we make LLMs faster and lighter? Don’t force the GPU to adapt to sparsity. Reshape the sparsity to fit the GPU! ⚡️ Excited to share our new #ICML2026 paper in collaboration with NVIDIA: “Sparser, Faster, Lighter Transformer Language Models”. This work introduces new open-source GPU kernels and data formats for faster inference and training of sparse transformer language models: Paper: https://arxiv.org/abs/2603.23198 Technical Blog: https://pub.sakana.ai/sparser-faster-llms Code: https://github.