Trendoday

How Much Training-Inference Mismatch Can On-Policy Distillation Tolerate?

On-policy distillation (OPD) trains a student language model on its own responses, guided by a fixed teacher. Generating these responses is expensive, so systems use quantized inference and asynchrono

  • Upvotes5
–Rank now
#12Peak rank
2Days on list
191Score · 7d
Nov 10, 2026First seen
Rank · 7 days

About this paper

On-policy distillation tolerates substantial training-inference mismatch from quantized inference and asynchronous rollouts, with model- and layer-specific limits, and a lightweight admission check plus FP8 recipes can exploit this tolerance to improve training efficiency.

On-policy distillation (OPD) trains a student language model on its own responses, guided by a fixed teacher. Generating these responses is expensive, so systems use quantized inference and asynchronous rollouts to reduce the cost. Both, however, introduce training-inference mismatch between the rollout engine and the model being trained. Such mismatch can destabilize reinforcement learning (RL), but its effects on OPD remain unclear. This paper investigates how much mismatch OPD tolerates, where its effects arise, and how training systems can use this tolerance. Measuring all mismatch sources on one scale, we examine quantization across three student-teacher pairs, with additional tests…

on-policy distillationstudent language modelteacher modelquantized inferenceasynchronous rolloutstraining-inference mismatchrollout enginereinforcement learningquantizationstudent-teacher pairs
Upvotes
5
Authors
8
Published
Nov 8, 2026

Summary and abstract excerpt via Hugging Face Papers ↗ · arXiv.

Trending history

How Much Training-Inference Mismatch Can On-Policy Distillation Tolerate? was first seen on Hugging Face · Daily papers on Nov 10, 2026. Since then it has appeared in 14 checks on 2 different days. Its best position was #12 of 30, reached on Nov 11, 05:28 UTC. It dropped off the list after Nov 11, 11:28 UTC.

Rank history

Position on Hugging Face · Daily papers at every check (lower is better).

#1#11#20#30 09:2815:5822:2804:5811:28 Nov 10, 09:28 UTC · #29Nov 10, 11:28 UTC · #24Nov 10, 13:28 UTC · #19Nov 10, 15:28 UTC · #18Nov 10, 17:28 UTC · #15Nov 10, 19:28 UTC · #15Nov 10, 21:28 UTC · #14Nov 10, 23:28 UTC · #14Nov 11, 01:28 UTC · #14Nov 11, 03:28 UTC · #14Nov 11, 05:28 UTC · #12Nov 11, 07:28 UTC · #17Nov 11, 09:28 UTC · #17Nov 11, 11:28 UTC · #16

Last 14 days

Best rank per day (UTC).

Nov 3today

Metric history

Platform metric as reported by Hugging Face.

04812 09:2815:5822:2804:5811:28 Nov 10, 09:28 UTC · 5Nov 10, 11:28 UTC · 6Nov 10, 13:28 UTC · 7Nov 10, 15:28 UTC · 8Nov 10, 17:28 UTC · 10Nov 10, 19:28 UTC · 11Nov 10, 21:28 UTC · 11Nov 10, 23:28 UTC · 11Nov 11, 01:28 UTC · 11Nov 11, 03:28 UTC · 11Nov 11, 05:28 UTC · 11Nov 11, 07:28 UTC · 11Nov 11, 09:28 UTC · 11Nov 11, 11:28 UTC · 12

Questions

Is How Much Training-Inference Mismatch Can On-Policy Distillation Tolerate? trending on HF Daily Papers right now?

Not at the moment. It was last on the list on Nov 11, 11:28 UTC, after 2 days of trending.

What is the highest rank How Much Training-Inference Mismatch Can On-Policy Distillation Tolerate? reached?

#12 on Hugging Face · Daily papers, on Nov 11, 05:28 UTC. Trendoday checks this list every 120 minutes.

How long has How Much Training-Inference Mismatch Can On-Policy Distillation Tolerate? been trending?

It was first seen on Nov 10, 2026 and has been on the list on 2 days (14 checks) since.