Mixture-of-Experts (MoE) models route each token to a few of many experts, yet what an individual expert is responsible for remains unsettled. Existing analyses count how often an expert is selected o
A sparse autoencoder trained on mixture-of-experts layer outputs reveals that individual experts drive specific latent features, showing finer-grained and more concentrated specialization than routing or domain labels alone.
Mixture-of-Experts (MoE) models route each token to a few of many experts, yet what an individual expert is responsible for remains unsettled. Existing analyses count how often an expert is selected on domain-labeled corpora, an instrument that fixes the taxonomy in advance, stays coarse, and records selection rather than contribution. We instead study specialization in feature space by training a sparse autoencoder (SAE) on the output of an MoE layer. Since this output is a router-weighted sum of expert outputs and the SAE encoder is affine, the pre-activation of every latent decomposes exactly over the experts. Aggregated over a corpus, this gives each expert a dominant set of latents…
Summary and abstract excerpt via Hugging Face Papers ↗ · arXiv.
SAE Interprets MoE: Sparse AutoEncoders Reveal That MoE Experts Exhibit Human-Interpretable Fine-Grained Specialization was first seen on Hugging Face · Daily papers on Nov 6, 2026. Since then it has appeared in 26 checks on 3 different days. Its best position was #24 of 30, reached on Nov 7, 09:28 UTC. It is #25 right now, holding steady at #25 over the last 6 checks.
Position on Hugging Face · Daily papers at every check (lower is better).
Best rank per day (UTC).
▲ as reported by Hugging Face.
Archived daily, weekly and monthly lists that include this entry.
Yes. As of Nov 8, 17:28 UTC it is #25, holding steady at #25 over the last 6 checks.
#24 on Hugging Face · Daily papers, on Nov 7, 09:28 UTC. Trendoday checks this list every 120 minutes.
It was first seen on Nov 6, 2026 and has been on the list on 3 days (26 checks) since.