Multi-teacher on-policy distillation (MOPD) extends on-policy distillation (OPD) to consolidate capabilities from multiple teachers. Although prior work reports near-perfect in-distribution (IND) dist
Multi-teacher on-policy distillation (MOPD) extends on-policy distillation (OPD) to consolidate capabilities from multiple teachers. Although prior work reports near-perfect in-distribution (IND) distillation performance, we find that this can mask incomplete capability consolidation beyond the supervised distribution. A preliminary study on synthetic reasoning shows that OPD does not fully transfer teachers' generalization capabilities. We then extend this investigation to real-world tasks, evaluating extrapolative and compositional generalization across mathematical reasoning, coding, agent, and long-context capabilities using Qwen3 models. Across extrapolative settings, OPD and MOPD…
Summary and abstract excerpt via Hugging Face Papers ↗ · arXiv.
On the Generalization Limitation of On-Policy Distillation for Post-RL Consolidation was first seen on Hugging Face · Daily papers on Nov 5, 2026. Since then it has appeared in 11 checks on 2 different days. Its best position was #10 of 30, reached on Nov 6, 01:28 UTC. It dropped off the list after Nov 6, 15:28 UTC.
Position on Hugging Face · Daily papers at every check (lower is better).
Best rank per day (UTC).
Platform metric as reported by Hugging Face.
Archived daily, weekly and monthly lists that include this entry.
Not at the moment. It was last on the list on Nov 6, 15:28 UTC, after 2 days of trending.
#10 on Hugging Face · Daily papers, on Nov 6, 01:28 UTC. Trendoday checks this list every 120 minutes.
It was first seen on Nov 5, 2026 and has been on the list on 2 days (11 checks) since.