1
MuLoCo: Muon is a practical inner optimizer for DiLoCo
DiLoCo在大模型训练中性能退化，改用Muon优化器后效果显著提升，为分布式训练提供新思路。
arXiv:2505.23725v3 Announce Type: replace Abstract: DiLoCo is a powerful framework for training large language models (LLMs), enabling larger optimal …