牛哥精选 · 本月

📋 全部 🤖 AI·大模型 ⚡ 效率工具 📝 深度技术 🚀 产品观察 💰 商业科技 🔓 开源项目 🎨 设计创意 📖 阅读推荐 🏷 资源合集 🌱 成长效率

📝 深度技术 arXiv NLP 2026-06-05

Predictable Scaling Laws of Optimal Hyperparameters for LLM Continued Pre-training

LLM继续预训练中，超参数配置可预测的缩放规律，告别启发式搜索与高昂成本

arXiv:2606.05610v1 Announce Type: new Abstract: The efficacy of continued pre-training for Large Language Models (LLMs) hinges upon hyperparameter con…

llm 持续预训练超参数缩放规律训练优化

📝 深度技术 arXiv 机器学习 2026-06-02

When Data Is Scarce: Scaling Sparse Language Models with Repeated Training

数据稀缺时如何扩展稀疏语言模型？这篇ICML 2026论文提出重复训练方法，有望突破数据瓶颈。

arXiv:2606.01155v1 Announce Type: new Abstract: Scaling laws for dense LLMs under infinite data are well explored, but how sparsity interacts with lim…

稀疏语言模型数据稀缺重复训练缩放规律 icml2026

📅 日期

2026-05-20 2026-05-19

🐂 牛哥精选

Predictable Scaling Laws of Optimal Hyperparameters for LLM Continued Pre-training

When Data Is Scarce: Scaling Sparse Language Models with Repeated Training

📅 日期