Raptor integrates logic with 3D-stacked DRAM to attack generative inference bandwidth limits

Early Raptor silicon combines accelerator logic with face-to-face stacked DRAM and reports up to 100 TB/s per card. Across six language and speech workloads, the paper reports 4.71 times the throughput of HBM designs and 2.44 times that of SRAM designs under its modeled deployment assumptions.