探讨SOAP、Muon等新型优化器如何突破LLM预训练规模瓶颈，13位作者揭示前沿缩放规律
arXiv:2607.20548v1 Announce Type: cross Abstract: Higher-order optimizers such as Muon and SOAP offer faster convergence than AdamW, but their computa…
Muon优化器比Adam学习到更鲁棒和可迁移的特征，为深度学习训练提供新选择。
arXiv:2606.09658v1 Announce Type: new Abstract: Muon has recently emerged as a state-of-the-art optimizer for pretraining Large Language Models (LLMs)…
DiLoCo在大模型训练中性能退化，改用Muon优化器后效果显著提升，为分布式训练提供新思路。
arXiv:2505.23725v3 Announce Type: replace Abstract: DiLoCo is a powerful framework for training large language models (LLMs), enabling larger optimal …
Muon优化器正交化程度对训练效果的影响研究，给出放松的立方Newton-Schulz调度方案
arXiv:2606.00371v1 Announce Type: new Abstract: Muon optimizers improve neural-network training by replacing ill-conditioned momentum updates with app…
MuCon通过裁剪奇异值改进Muon优化器，为LLM训练提供更稳定的更新策略
arXiv:2605.26459v1 Announce Type: new Abstract: Muon-style optimizers take a matrix-valued momentum or preconditioned update $B = U \operatorname{diag…
Muon优化器新升级MONA，融合Nesterov加速突破局部极小，提升大模型训练效率
arXiv:2605.26842v1 Announce Type: new Abstract: The Muon optimizer has recently offered a promising alternative to AdamW for large language model trai…
Muon优化器新突破，基于重尾谱校正解决噪声方向过量问题，助力大模型高效训练
arXiv:2603.10067v2 Announce Type: replace-cross Abstract: Muon has recently shown promising results in LLM training. In this work, we study how to fur…
Muon优化器新变体AMUSE，实现任意时刻稳定梯度评估，41页论文含25图详解算法改进。
arXiv:2605.22432v1 Announce Type: new Abstract: Modern deep learning commonly relies on AdamW with prescribed learning rate schedules, but recent work…
深入解读从SGD到Muon的优化器演进，以Schatten-p范数统一矩阵几何约束，为AI研究者提供理论新视角
arXiv:2605.19781v1 Announce Type: new Abstract: Modern optimizers, like Muon, impose matrix-wise geometry constraints on their updates. These matrix-w…
Muon优化器在视觉语言对齐与强化学习微调中暴露频谱失效问题，作者提出高通滤波器补救方案，刷新大模型训练认知。
arXiv:2605.19282v1 Announce Type: new Abstract: Muon is a matrix-aware optimizer that leverages Newton-Schulz (NS) iterations to enforce spectral grad…
提出自适应Muon正交化方法，有望优化深度学习训练过程。
arXiv:2605.17806v1 Announce Type: new Abstract: Muon has recently emerged as a competitive alternative to AdamW for large-scale pre-training, with ort…
这篇论文提出了DynMuon，从动态谱成形视角重新理解Muon优化器，为训练大模型提供理论洞察与方法改进。
arXiv:2605.17109v1 Announce Type: new Abstract: In recent years, Muon has emerged as the dominant method for training large language models, and trans…