DiffusionBlocks: Training Neural Networks One Block at a Time

–> TL;DR For over a decade, we’ve accepted that end-to-end backprop is the only way to train deep networks. But holding the entire network in memory all at once is why AI training is hitting a resource wall. We found a new way to break the network into blocks and train them independently.