1
$M^3$ Scaling Law: Optimizing Multi-Epoch, Multi-Lingual, and Multi-Stage Training for Low-Resource Language Models
揭示低资源语言模型在多轮、多语言、多阶段训练中的新缩放定律，为高效预训练提供理论指导与实践依据。
arXiv:2410.12325v2 Announce Type: replace Abstract: In this paper, we study a fundamental design problem in pretraining Large Language Models (LLMs) f…