Answer-level rewards cannot distinguish reasoning traces with equally correct outcomes. We study whether auxiliary trajectory preferences improve small-language-model self-improvement and transfer und
Auxiliary trajectory preferences based on entropy and confidence metrics improve small language model self-improvement and transfer by distinguishing learning value among equally correct reasoning traces.
Answer-level rewards cannot distinguish reasoning traces with equally correct outcomes. We study whether auxiliary trajectory preferences improve small-language-model self-improvement and transfer under limited supervision. We compare confidence-based (SC) and entropy-based metrics (UID-Conj.), which combines global entropy variation and local smoothness, through online reward augmentation and offline training on correct responses. Online experiments adapt Qwen3-1.7B on 30 AIME problems without reference answers; offline experiments use historical AIME problems. UID-Conj. improves the online macro-average across the adaptation benchmark and three transfer benchmarks by 2.08 percentage…
Summary and abstract excerpt via Hugging Face Papers ↗ · arXiv.
Self-Improvement with Auxiliary Rewards: Learning from Verified Traces and Evaluating Transfer was first seen on Hugging Face · Daily papers on Nov 3, 2026. Since then it has appeared in 1 check on 1 different day. Its best position was #29 of 30, reached on Nov 3, 15:28 UTC. It dropped off the list after Nov 3, 15:28 UTC.
Position on Hugging Face · Daily papers at every check (lower is better).
Best rank per day (UTC).
Archived daily, weekly and monthly lists that include this entry.
Not at the moment. It was last on the list on Nov 3, 15:28 UTC, after 1 day of trending.
#29 on Hugging Face · Daily papers, on Nov 3, 15:28 UTC. Trendoday checks this list every 120 minutes.
It was first seen on Nov 3, 2026 and has been on the list on 1 day (1 check) since.