Ask to Distill: Learning to Synthesize Data for On-Policy Distillation
On-policy distillation (OPD) can succeed with surprisingly few training questions, motivating a view of questions as carriers of distributional knowledge transfer. How can we assess whether a newly ge