1
Robust and Fast Training via Per-Sample Clipping
逐样本裁剪的稳健梯度估计，同时提速训练并增强鲁棒性，值得一试。
arXiv:2605.02701v2 Announce Type: replace-cross Abstract: We propose a robust gradient estimator based on per-sample gradient clipping and analyze its…
2
Quartet II: Accurate LLM Pre-Training in NVFP4 by Improved Unbiased Gradient Estimation
NVFP4精度下的LLM预训练新突破，创新梯度估计方法显著提升模型准确性。
arXiv:2601.22813v2 Announce Type: replace Abstract: The NVFP4 lower-precision format, supported in hardware by NVIDIA Blackwell GPUs, promises to allo…