Regularized Score Centering
Asynchronous reinforcement learning increases training throughput, but data from stale policies and numerical mismatch between training and inference can destabilize learning. To stabilize off-policy
- Upvotes7
Asynchronous reinforcement learning increases training throughput, but data from stale policies and numerical mismatch between training and inference can destabilize learning. To stabilize off-policy
Asynchronous reinforcement learning increases training throughput, but data from stale policies and numerical mismatch between training and inference can destabilize learning. To stabilize off-policy learning, score centering (SC) removes the effect of reward baselines on the expected update. Our gradient diagnostics show less extreme gradients with SC than with importance sampling (IS), but SC can still fail when the learner moves far from the behavior policy that generated the data. To limit this policy mismatch, we introduce Regularized Score Centering (RSC), which adds a reverse Kullback-Leibler (KL) penalty to encourage the learner's next-token distribution to stay close to the…
Summary and abstract excerpt via Hugging Face Papers ↗ · arXiv.
Regularized Score Centering was first seen on Hugging Face · Daily papers on Nov 18, 2026. Since then it has appeared in 4 checks on 2 different days. Its best position was #20 of 30, reached on Nov 18, 07:28 UTC. It dropped off the list after Nov 19, 01:28 UTC.
Position on Hugging Face · Daily papers at every check (lower is better).
Best rank per day (UTC).
Platform metric as reported by Hugging Face.
Archived daily, weekly and monthly lists that include this entry.
Not at the moment. It was last on the list on Nov 19, 01:28 UTC, after 2 days of trending.
#20 on Hugging Face · Daily papers, on Nov 18, 07:28 UTC. Trendoday checks this list every 120 minutes.
It was first seen on Nov 18, 2026 and has been on the list on 2 days (4 checks) since.