牛哥精选 · 所有

📋 全部 🤖 AI·大模型 ⚡ 效率工具 📝 深度技术 🚀 产品观察 💰 商业科技 🔓 开源项目 🎨 设计创意 📖 阅读推荐 🏷 资源合集 🌱 成长效率

📝 深度技术 arXiv 机器学习 2026-07-13

Complexity-Guided Component-wise Initialization for Language Model Pretraining

巧妙利用预训练模型权重谱的结构化模式作为初始化信号，提升GPT-2风格语言模型预训练效率

arXiv:2607.09204v1 Announce Type: cross Abstract: Pretrained language models often exhibit structured weight spectra, suggesting that training may rep…

语言模型预训练初始化方法权重谱 gpt-2 结构模式

📝 深度技术 arXiv AI 2026-05-19

Pretraining large language models with MXFP4 on Native FP4 Hardware

研究揭示大模型全FP4训练发散根源，通过控制实验逐步启用MXFP4量化，为高效低精度训练提供新思路。

arXiv:2605.09825v3 Announce Type: replace-cross Abstract: Why does full-pipeline FP4 training of large language models often diverge, even when forwar…

mxfp4 fp4硬件大语言模型预训练量化训练模型训练稳定性

📅 日期

2026-05-20 2026-05-19

🐂 牛哥精选

Complexity-Guided Component-wise Initialization for Language Model Pretraining

Pretraining large language models with MXFP4 on Native FP4 Hardware

📅 日期