Channel Selection for Mixture-of-Experts Decoding Needs No Calibration Data
Mixture-of-Experts (MoE) models are bottlenecked by loading routed-expert weights from memory during interactive decoding at small batch sizes. Because most intermediate channels in a routed expert co
- Upvotes21