TensorRT-LLM refused FP8 block-scale checkpoints with non-gated experts on GB10, the chip inside DGX Spark. Our pull request makes them run: 64.3% more throughput and 44.9% less weight memory than BF16, with accuracy unchanged.
Need help?
Contact usTensorRT-LLM refused FP8 block-scale checkpoints with non-gated experts on GB10, the chip inside DGX Spark. Our pull request makes them run: 64.3% more throughput and 44.9% less weight memory than BF16, with accuracy unchanged.