About this paper
P$^2$-MOPD jointly optimizes mixing coefficients and student weights under an on-policy distillation objective, allowing experts to act as parameter priors and achieving faster, more balanced recovery than MOPD.
Multi-teacher on-policy distillation consolidates expert capabilities by supervising student-generated trajectories, whereas task arithmetic offers a training-free alternative through linear parameter combinations. We view the linear combination of student and expert parameters as a single differentiable model and propose P^2-MOPD, which jointly optimizes the mixing coefficients and the student weights under the same on-policy distillation objective. Unlike task arithmetic, which stays within the span of the frozen student and expert parameters for any choice of merging coefficients, P^2-MOPD keeps the student trainable, so the experts act as parameter priors without confining it to this…
multi-teacher on-policy distillationstudent-generated trajectoriestask arithmeticlinear parameter combinationsdifferentiable modelP$^2$-MOPDmixing coefficientsstudent weightson-policy distillation objectivefrozen student and expert parameters
- Upvotes
- 9
- Authors
- 11
- Published
- Nov 2, 2026
Summary and abstract excerpt via Hugging Face Papers ↗ · arXiv.
Trending history
P^2-MOPD: Accelerating Multi-Teacher On-Policy Distillation via Parameter Priors was first seen on Hugging Face · Daily papers on Nov 9, 2026. Since then it has appeared in 18 checks on 2 different days. Its best position was #3 of 30, reached on Nov 10, 01:28 UTC. It dropped off the list after Nov 10, 17:28 UTC.