On-policy Self-distillation (OPSD) has proven effective for post-training large language models by constructing a teacher conditioned on privileged references. However, existing approaches keep the te
Multi-teacher Adaptive On-policy Self-Distillation selects a preferred teacher for each token during training, improving mathematical reasoning performance over fixed-teacher methods.
On-policy Self-distillation (OPSD) has proven effective for post-training large language models by constructing a teacher conditioned on privileged references. However, existing approaches keep the teacher fixed either throughout training by consistently exposing it to the full reference, or within each response by adaptively varying partial reference exposure only across responses. In this paper, we argue that neither of them is sufficient. Instead, every token needs its own teacher, as the teacher preference can vary token by token within the same response. Through a teacher-intervention pilot study, we validate that the teacher preference differs across tokens, and reveal that a…
Summary and abstract excerpt via Hugging Face Papers ↗ · arXiv.
Every Token Needs Its Own Teacher: Multi-teacher Adaptive On-policy Self-distillation was first seen on Hugging Face · Daily papers on Nov 4, 2026. Since then it has appeared in 20 checks on 3 different days. Its best position was #14 of 30, reached on Nov 5, 07:28 UTC. It dropped off the list after Nov 6, 03:28 UTC.
Position on Hugging Face · Daily papers at every check (lower is better).
Best rank per day (UTC).
Platform metric as reported by Hugging Face.
Archived daily, weekly and monthly lists that include this entry.
Not at the moment. It was last on the list on Nov 6, 03:28 UTC, after 3 days of trending.
#14 on Hugging Face · Daily papers, on Nov 5, 07:28 UTC. Trendoday checks this list every 120 minutes.
It was first seen on Nov 4, 2026 and has been on the list on 3 days (20 checks) since.