1
Soft-to-Hard Routing in Sparse Mixture-of-Experts Models
提出一种从软路由过渡到硬路由的机制,在稀疏MoE中同时优化模型性能与计算效率。
arXiv:2605.02124v2 Announce Type: replace-cross Abstract: Softmax routing approaches hard top-1 routing as the temperature tends to zero, but the limi…