Attention remains a substantial computational cost in Transformer training, particularly at long context lengths. NVIDIA Blackwell GPUs provide high-throughput low-precision matrix multiplication, but
FLAMIA accelerates Transformer attention on NVIDIA Blackwell GPUs via MXFP8 and mixed-precision kernels that reduce quantization overhead, prevent causal leakage, and support variable-length packed documents.
Attention remains a substantial computational cost in Transformer training, particularly at long context lengths. NVIDIA Blackwell GPUs provide high-throughput low-precision matrix multiplication, but exploiting it in attention requires addressing quantization overhead, causal information leakage through shared scales, and numerical inconsistencies between the forward and backward passes. We present FLAMIA, a family of attention kernels for Blackwell with MXFP8 and mixed NVFP4/FP8 training paths. Our design combines token-block scaling that reuses quantized operands across matrix orientations, targeted safeguards against scale-induced leakage, and consistent forward and backward…
Summary and abstract excerpt via Hugging Face Papers ↗ · arXiv.
FLAMIA: FLAsh MIcroscaling Attention for Fast and Causally Safe Training on Blackwell was first seen on Hugging Face · Daily papers on Nov 3, 2026. Since then it has appeared in 10 checks on 2 different days. Its best position was #11 of 30, reached on Nov 4, 05:28 UTC. It dropped off the list after Nov 4, 05:28 UTC.
Position on Hugging Face · Daily papers at every check (lower is better).
Best rank per day (UTC).
Platform metric as reported by Hugging Face.
Archived daily, weekly and monthly lists that include this entry.
Not at the moment. It was last on the list on Nov 4, 05:28 UTC, after 2 days of trending.
#11 on Hugging Face · Daily papers, on Nov 4, 05:28 UTC. Trendoday checks this list every 120 minutes.
It was first seen on Nov 3, 2026 and has been on the list on 2 days (10 checks) since.