Embedding models compress variable-length text into fixed-size vector representations, but this compression can entangle semantic content with residual token-level information, such as artifacts arisi
Prepending trainable [DUMP] tokens during embedding model training reduces residual token-level signals and improves retrieval representations without changing architecture, pooling, or loss.
Embedding models compress variable-length text into fixed-size vector representations, but this compression can entangle semantic content with residual token-level information, such as artifacts arising from attention sink behavior. We propose a simple method that prepends trainable [DUMP] tokens to the input during embedding model training. The method keeps the model architecture, pooling strategy, and training loss unchanged, while optimizing the added tokens through the standard contrastive retrieval objective. By introducing trainable prefix positions that interact with regular tokens during encoding, [DUMP] tokens help reduce the influence of residual or uninformative token-level…
Summary and abstract excerpt via Hugging Face Papers ↗ · arXiv.
Reducing Representational Interference in Retrieval Embeddings with Trainable Prefix Tokens was first seen on Hugging Face · Daily papers on Nov 7, 2026. Since then it has appeared in 7 checks on 2 different days. Its best position was #30 of 30, reached on Nov 7, 17:28 UTC. It dropped off the list after Nov 8, 05:28 UTC.
Position on Hugging Face · Daily papers at every check (lower is better).
Best rank per day (UTC).
Platform metric as reported by Hugging Face.
Archived daily, weekly and monthly lists that include this entry.
Not at the moment. It was last on the list on Nov 8, 05:28 UTC, after 2 days of trending.
#30 on Hugging Face · Daily papers, on Nov 7, 17:28 UTC. Trendoday checks this list every 120 minutes.
It was first seen on Nov 7, 2026 and has been on the list on 2 days (7 checks) since.