On-policy distillation (OPD) trains a student on its own generations by matching the teacher's token probabilities. These probabilities entangle two kinds of information: which tokens the teacher pref
Order-only on-policy distillation trains students to match teacher token rankings rather than probabilities, improving reasoning performance without temperature tuning.
On-policy distillation (OPD) trains a student on its own generations by matching the teacher's token probabilities. These probabilities entangle two kinds of information: which tokens the teacher prefers, and how confident it is. The latter reflects the teacher's own capacity and calibration, which a smaller student may be unable to reproduce. Because the teacher temperature changes these values without changing the ordering, OPD is also sensitive to the temperature choice. We ask whether the teacher's ordering of candidate tokens is enough, and propose Order-Only On-Policy Distillation (O^{3}PD). At each student-generated prefix, O^{3}PD trains the student to reproduce the order of the…
Summary and abstract excerpt via Hugging Face Papers ↗ · arXiv.
Order-Only On-Policy Distillation was first seen on Hugging Face · Daily papers on Nov 3, 2026. Since then it has appeared in 13 checks on 2 different days. Its best position was #5 of 30, reached on Nov 4, 05:28 UTC. It dropped off the list after Nov 4, 17:28 UTC.
Position on Hugging Face · Daily papers at every check (lower is better).
Best rank per day (UTC).
Platform metric as reported by Hugging Face.
Archived daily, weekly and monthly lists that include this entry.
Not at the moment. It was last on the list on Nov 4, 17:28 UTC, after 2 days of trending.
#5 on Hugging Face · Daily papers, on Nov 4, 05:28 UTC. Trendoday checks this list every 120 minutes.
It was first seen on Nov 3, 2026 and has been on the list on 2 days (13 checks) since.