Reinforcement learning objectives for diffusion models are reformulated for few-step samplers through x-space transformation, gradient decomposition, and anchor adaptation, yielding modular few-step variants of several methods and showing reward-gradient methods excel with differentiable rewards…
Reinforcement learning (RL) has become a standard post-training method for diffusion models, but most RL objectives roll out the current policy at every iteration, and with a many-step sampler these rollouts typically dominate the cost of training. Few-step models cut this cost, yet they model an average velocity over an interval, whereas existing RL objectives are built for an instantaneous velocity with a many-step sampler. We adapt RL objectives to few-step models systematically in three phases, namely a transformation, a decomposition and an adaptation. The transformation to x-space expresses each objective we consider through a common policy prediction, at the clean end by default and…
reinforcement learningdiffusion modelspost-trainingmany-step samplerfew-step modelsaverage velocityinstantaneous velocityx-spacepolicy predictionflow-map objectives
- Upvotes
- 4
- Authors
- 7
- Published
- Nov 5, 2026
Summary and abstract excerpt via Hugging Face Papers ↗ · arXiv.