Linear attention compresses context into a fixed-size key-value state for linear-time training and constant-memory inference. Gated delta-rule models decay this state and erase its prediction for an i
A linear attention architecture called Muir allocates gate capacity by sensitivity, uses matched-filter readout and a stable parallel update, improving language modeling, long-context recall, and training speed over Gated DeltaNet-2 and Transformer baselines.
Linear attention compresses context into a fixed-size key-value state for linear-time training and constant-memory inference. Gated delta-rule models decay this state and erase its prediction for an incoming key before writing a new value; Gated DeltaNet-2 (GDN-2) generalizes this with channel-wise decay and decoupled, full-rank erase and write gates. However, we show that state updates are far less sensitive to erasure than to writes: key activation and unit normalization concentrate key energy onto a sparse subset of channels, whereas value writes scale all channels, so the erase gate can be parameterized at lower rank. Furthermore, slow-decaying state rows accumulate much larger norms…
Summary and abstract excerpt via Hugging Face Papers ↗ · arXiv.
Muir: Linear Attention with Sensitivity-Allocated Gating and Matched-Filter Readout was first seen on Hugging Face · Daily papers on Nov 6, 2026. Since then it has appeared in 23 checks on 3 different days. Its best position was #20 of 30, reached on Nov 8, 11:28 UTC. It is #20 right now, climbing from #24 to #20 over the last 6 checks.
Position on Hugging Face · Daily papers at every check (lower is better).
Best rank per day (UTC).
▲ as reported by Hugging Face.
Archived daily, weekly and monthly lists that include this entry.
Yes. As of Nov 8, 17:28 UTC it is #20, climbing from #24 to #20 over the last 6 checks.
#20 on Hugging Face · Daily papers, on Nov 8, 11:28 UTC. Trendoday checks this list every 120 minutes.
It was first seen on Nov 6, 2026 and has been on the list on 3 days (23 checks) since.