Sparse attention cuts the cost of long-context inference by making the attention matrix sparse: each query attends only a subset of the sequence, chosen by a fixed pattern or, in recent production LLM
A multi-layer indexer reuses adjacent layers’ token selections to overlap indexer scans with attention, improving long-context decode throughput when combined with cross-layer indexer reuse while maintaining accuracy.
Sparse attention cuts the cost of long-context inference by making the attention matrix sparse: each query attends only a subset of the sequence, chosen by a fixed pattern or, in recent production LLMs, learned per query. In DeepSeek Sparse Attention (DSA), the attention of DeepSeek-V3.2 and the GLM-5 family, every layer is preceded by a lightning indexer that scores the whole cache and keeps the k=2048 highest-scoring tokens for the layer to attend. At long context this indexer scan, not the sparse attention it feeds, is where the decode time goes. But adjacent layers select nearly the same tokens, so a layer can attend the selection of the previous indexer layer instead of its own. This…
Summary and abstract excerpt via Hugging Face Papers ↗ · arXiv.
Multi-layer Indexer: Overlapping Selection and Attention in DeepSeek Sparse Attention was first seen on Hugging Face · Daily papers on Nov 7, 2026. Since then it has appeared in 4 checks on 1 different day. Its best position was #30 of 30, reached on Nov 7, 01:28 UTC. It dropped off the list after Nov 7, 07:28 UTC.
Position on Hugging Face · Daily papers at every check (lower is better).
Best rank per day (UTC).
Platform metric as reported by Hugging Face.
Archived daily, weekly and monthly lists that include this entry.
Not at the moment. It was last on the list on Nov 7, 07:28 UTC, after 1 day of trending.
#30 on Hugging Face · Daily papers, on Nov 7, 01:28 UTC. Trendoday checks this list every 120 minutes.
It was first seen on Nov 7, 2026 and has been on the list on 1 day (4 checks) since.