AdamW在重尾噪声下是否有效？这篇论文提出了一个亟待解决的开放问题。
arXiv:2606.23676v1 Announce Type: cross Abstract: AdamW is the de facto optimizer for training large language models (LLMs), yet the theory behind it …
Muon优化器比Adam学习到更鲁棒和可迁移的特征，为深度学习训练提供新选择。
arXiv:2606.09658v1 Announce Type: new Abstract: Muon has recently emerged as a state-of-the-art optimizer for pretraining Large Language Models (LLMs)…
基于E8晶格的新型LLM量化方法GLQ，8权重一组映射为16位索引，精度2-8 bpw，开源有代码。
I have with the help of AI create an open source method of E8 LLM code book quantization library called glq. I was interested in creating Glq as a PC …
揭秘SGD在LLM预训练中不如Adam的根源：大有效学习率的关键作用。
arXiv:2605.17787v1 Announce Type: new Abstract: It is widely believed that stochastic gradient descent (SGD) performs significantly worse than adaptiv…