About this paper
Distilled few-step diffusion models exhibit diversity collapse because activation variance is concentrated into fewer directions in an intermediate block at the initial denoising step, and ReDiverse restores sample diversity by reintroducing teacher variance directions through adapter fine-tuning…
Distilling diffusion models into few-step student models substantially reduces sampling latency, but induces severe diversity collapse: samples generated from different random seeds show minimal variation. In this work, we investigate the root cause of this failure by probing the internal representations of student models relative to their teachers. We find that distilled students maintain total activation variance comparable to their teachers, but concentrate it into substantially fewer directions. This directional collapse is localized to an identifiable intermediate block at the initial denoising step. Consequently, diversity collapse stems from a misallocation of activation variance…
diffusion modelsfew-step student modelsdiffusion model distillationdiversity collapseinternal representationsactivation variancedirectional collapseintermediate blockinitial denoising stepReDiverse
- Upvotes
- 4
- Authors
- 7
- Published
- Nov 13, 2026
Summary and abstract excerpt via Hugging Face Papers ↗ · arXiv.
Trending history
Restoring Diversity in Distilled Image and Video Generators by Probing Activations was first seen on Hugging Face · Daily papers on Nov 17, 2026. Since then it has appeared in 7 checks on 2 different days. Its best position was #23 of 30, reached on Nov 17, 23:28 UTC. It dropped off the list after Nov 18, 01:28 UTC.