Self-distillation methods for large language models with an exponential moving average (EMA) teacher already retain a history of student updates. Muon nevertheless stores a separate momentum buffer, i
Muon-RT eliminates separate momentum buffers by reusing teacher-student displacement for hidden weight matrices, reducing memory while improving fine-tuning and policy optimization accuracy.
Self-distillation methods for large language models with an exponential moving average (EMA) teacher already retain a history of student updates. Muon nevertheless stores a separate momentum buffer, increasing persistent training memory. We propose Muon with Reused Teacher (Muon-RT), which reuses the teacher-student displacement to eliminate momentum buffers for hidden weight matrices. We show that this displacement encodes an exponentially weighted history of applied updates and smooths temporally uncorrelated noise under a local model. Muon-RT removes its radial component, normalizes it spectrally, and mixes it with the current gradient after Newton-Schulz (NS) orthogonalization. Under…
Summary and abstract excerpt via Hugging Face Papers ↗ · arXiv.
MUON-RT Reusing the Teacher's Trajectory for Momentum-Free Muon for LLM Self-Distillation was first seen on Hugging Face · Daily papers on Nov 5, 2026. Since then it has appeared in 16 checks on 2 different days. Its best position was #11 of 30, reached on Nov 6, 03:28 UTC. It dropped off the list after Nov 6, 13:28 UTC.
Position on Hugging Face · Daily papers at every check (lower is better).
Best rank per day (UTC).
Platform metric as reported by Hugging Face.
Archived daily, weekly and monthly lists that include this entry.
Not at the moment. It was last on the list on Nov 6, 13:28 UTC, after 2 days of trending.
#11 on Hugging Face · Daily papers, on Nov 6, 03:28 UTC. Trendoday checks this list every 120 minutes.
It was first seen on Nov 5, 2026 and has been on the list on 2 days (16 checks) since.