Locating relevant evidence in long inputs is a core capability of Multimodal Large Language Models. Prior studies attribute this capability to retrieval heads, a sparse set of attention heads that tra
Retrieval heads remain sparse and shared across audio-visual modalities, and optimizing them improves document retrieval performance in multimodal models.
Locating relevant evidence in long inputs is a core capability of Multimodal Large Language Models. Prior studies attribute this capability to retrieval heads, a sparse set of attention heads that transfer information from the context to the output. However, these analyses are confined to a limited set of modalities and the corresponding models. Moreover, how retrieval heads can be leveraged to enhance the model calls for further exploration. In this paper, we ask whether this mechanism expands together with the input modalities of the model. Taking audio-visual input as a newly added modality, we construct paired inputs in which the same answer is either spoken or shown on screen. With…
Summary and abstract excerpt via Hugging Face Papers ↗ · arXiv.
A Broader Look into MLLM's Retrieval Heads was first seen on Hugging Face · Daily papers on Oct 30, 2026. Since then it has appeared in 38 checks on 4 different days. Its best position was #6 of 30, reached on Nov 1, 11:28 UTC. It dropped off the list after Nov 2, 07:28 UTC.
Position on Hugging Face · Daily papers at every check (lower is better).
Best rank per day (UTC).
Platform metric as reported by Hugging Face.
Archived daily, weekly and monthly lists that include this entry.
Not at the moment. It was last on the list on Nov 2, 07:28 UTC, after 4 days of trending.
#6 on Hugging Face · Daily papers, on Nov 1, 11:28 UTC. Trendoday checks this list every 120 minutes.
It was first seen on Oct 30, 2026 and has been on the list on 4 days (38 checks) since.