Linear attention scales linearly with sequence length, but its finite-state memory can limit long-context retrieval. In gated linear recurrences, memory retention depends on state decay, which is shap
Dual-Axis Memory Initialization improves long-context retrieval in gated linear recurrent models by jointly configuring decay and sensitivity parameters before training.
Linear attention scales linearly with sequence length, but its finite-state memory can limit long-context retrieval. In gated linear recurrences, memory retention depends on state decay, which is shaped jointly by gate parameters A_log and dt_bias and by input activations. In controlled Gated DeltaNet (GDN) experiments, both parameters remain highly correlated with their initial values after training, highlighting the importance of initialization. Therefore, we introduce Dual-Axis Memory Initialization (DAMI), which jointly configures these parameters before training. DAMI assigns geometric timescales across heads for varied retrieval distances and a low-high-low input-sensitivity profile…
Summary and abstract excerpt via Hugging Face Papers ↗ · arXiv.
DAMI: Dual-Axis Memory Initialization for Enhanced Long-Context Retrieval in Gated Linear Attention was first seen on Hugging Face · Daily papers on Nov 4, 2026. Since then it has appeared in 2 checks on 1 different day. Its best position was #27 of 30, reached on Nov 4, 05:28 UTC. It dropped off the list after Nov 4, 11:28 UTC.
Position on Hugging Face · Daily papers at every check (lower is better).
Best rank per day (UTC).
Platform metric as reported by Hugging Face.
Archived daily, weekly and monthly lists that include this entry.
Not at the moment. It was last on the list on Nov 4, 11:28 UTC, after 1 day of trending.
#27 on Hugging Face · Daily papers, on Nov 4, 05:28 UTC. Trendoday checks this list every 120 minutes.
It was first seen on Nov 4, 2026 and has been on the list on 1 day (2 checks) since.