最新研究提出神经网络训练中的自动稳定性与恢复方法，为模型训练失败提供智能解决方案。
arXiv:2601.17483v2 Announce Type: replace-cross Abstract: Training modern neural networks is increasingly fragile, with rare but severe destabilizing …
提出归一化奖励方法，提升偏好优化训练稳定性与效果
arXiv:2607.16240v1 Announce Type: cross Abstract: Direct Alignment Algorithms (DAAs) such as DPO have become a common way to post-train and align LLMs…
提出机制驱动监控器，抢先检测LLM训练不稳定，提升大模型训练可靠性
arXiv:2606.28116v1 Announce Type: new Abstract: Frontier large language model training consumes massive accelerator fleets and long wall-clock computa…
Q-Flow用流模型做策略，兼顾稳定性与表达力，为强化学习带来新思路。
arXiv:2605.13435v2 Announce Type: replace-cross Abstract: There is growing interest in utilizing flow-based models as decision-making policies in rein…
被ICML接收的自适应梯度裁剪方法，有效提升LLM预训练稳定性，AI训练优化的新突破
arXiv:2502.11034v3 Announce Type: replace Abstract: Loss spikes remain a persistent obstacle in large-scale language model pretraining. While previous…
用全局归一化稳定多模态大模型基于策略的蒸馏，提升推理性能与训练效率的创新方法。
arXiv:2606.09091v1 Announce Type: new Abstract: On-policy distillation (OPD) has recently emerged as an important post-training paradigm. By using a s…
突破PreNorm与PostNorm的困境：SpanNorm在提升深度Transformer训练稳定性同时保持高性能
arXiv:2601.22580v2 Announce Type: replace-cross Abstract: The success of Large Language Models (LLMs) hinges on the stable training of deep Transforme…
多项式预条件层PC Layer通过重塑权重矩阵奇异值谱，稳定大模型预训练过程，提升训练质量与收敛效率。
arXiv:2606.06470v1 Announce Type: cross Abstract: We propose a preconditioning (PC) layer, a weight parameterization via polynomial preconditioner tha…
用可微分最优传输优化MoE架构，提升大模型推理效率且训练更稳定。
arXiv:2606.01666v1 Announce Type: new Abstract: The scaling of Large Language Models (LLMs) has driven significant performance gains but created subst…
提出PowLU激活函数新方案，专门解决大语言模型预训练过程的不稳定性问题。
arXiv:2605.25704v1 Announce Type: cross Abstract: In contemporary large language models (LLMs), the swish-gated linear unit (SwiGLU) activation functi…
一种名为STAPO的新方法通过静默罕见虚假token来稳定LLM的强化学习训练，显著提升模型输出质量与收敛效率。
arXiv:2602.15620v5 Announce Type: replace-cross Abstract: Reinforcement Learning (RL) has significantly improved large language model reasoning, but e…
深度揭秘权重衰减对训练稳定性的真实作用，挑战传统正则化认知。
arXiv:2605.16622v1 Announce Type: new Abstract: In modern deep learning, weight decay is often credited with "stabilizing" training dynamics, divergin…
研究揭示大模型全FP4训练发散根源，通过控制实验逐步启用MXFP4量化，为高效低精度训练提供新思路。
arXiv:2605.09825v3 Announce Type: replace-cross Abstract: Why does full-pipeline FP4 training of large language models often diverge, even when forwar…