Attention at Absolute Zero
Attention scores candidate supports of an answer, which we call witnesses, and reads the scores at a temperature. We build a higher-order attention layer whose state lives on pairs of items: for a pai
- Upvotes3
Attention scores candidate supports of an answer, which we call witnesses, and reads the scores at a temperature. We build a higher-order attention layer whose state lives on pairs of items: for a pai
Attention scores candidate supports of an answer, which we call witnesses, and reads the scores at a temperature. We build a higher-order attention layer whose state lives on pairs of items: for a pair (a, b), each third item k is a witness judged through its legs (a, k) and (k, b). It reads witnesses at zero temperature, keeping the best and the number tied with it, passes on only bits comparing such counts, rebuilds its pair state from the input and these bits at every loop, and is trained from answers alone. Each choice removes one way soft attention fails at size: dispersed weight, unresolved gaps between learned scores, and a drifting soft state. In maximum retrieval it is right on at…
Summary and abstract excerpt via Hugging Face Papers ↗ · arXiv.
Attention at Absolute Zero was first seen on Hugging Face · Daily papers on Nov 19, 2026. Since then it has appeared in 3 checks on 1 different day. Its best position was #25 of 30, reached on Nov 19, 09:29 UTC. It dropped off the list after Nov 19, 09:29 UTC.
Position on Hugging Face · Daily papers at every check (lower is better).
Best rank per day (UTC).
Platform metric as reported by Hugging Face.
Archived daily, weekly and monthly lists that include this entry.
Not at the moment. It was last on the list on Nov 19, 09:29 UTC, after 1 day of trending.
#25 on Hugging Face · Daily papers, on Nov 19, 09:29 UTC. Trendoday checks this list every 120 minutes.
It was first seen on Nov 19, 2026 and has been on the list on 1 day (3 checks) since.