牛哥精选 · 本周

📋 全部 🤖 AI·大模型 ⚡ 效率工具 📝 深度技术 🚀 产品观察 💰 商业科技 🔓 开源项目 🎨 设计创意 📖 阅读推荐 🏷 资源合集 🌱 成长效率

🤖 AI·大模型 arXiv AI 2026-05-23

One LR Doesn't Fit All: Heavy-Tail Guided Layerwise Learning Rates for LLMs

突破传统统一学习率，重尾分布指导LLM逐层自适应学习，大幅提升训练效率与模型性能。

arXiv:2605.22297v1 Announce Type: cross Abstract: Learning rate configuration is a fundamental aspect of modern deep learning. The prevailing practice…

llm 学习率 heavy-tail 训练优化分层策略

📝 深度技术 arXiv 机器学习 2026-05-20

Revisiting the Adam-SGD Gap in LLM Pre-Training: The Role of Large Effective Learning Rates

揭秘SGD在LLM预训练中不如Adam的根源：大有效学习率的关键作用。

arXiv:2605.17787v1 Announce Type: new Abstract: It is widely believed that stochastic gradient descent (SGD) performs significantly worse than adaptiv…

llm预训练 adam优化器 sgd差距有效学习率深度学习

📝 深度技术 arXiv 机器学习 2026-05-20

Learning Rate Matters: Vanilla LoRA May Suffice for LLM Fine-tuning

通过调整学习率，简单LoRA即可媲美复杂微调方法，揭示被忽视的关键因素。

arXiv:2602.04998v2 Announce Type: replace Abstract: Low-Rank Adaptation (LoRA) is the prevailing approach for efficient large language model (LLM) fin…

lora 学习率大模型微调参数高效微调深度学习优化

📅 日期

2026-05-20 2026-05-19

🐂 牛哥精选

One LR Doesn't Fit All: Heavy-Tail Guided Layerwise Learning Rates for LLMs

Revisiting the Adam-SGD Gap in LLM Pre-Training: The Role of Large Effective Learning Rates

Learning Rate Matters: Vanilla LoRA May Suffice for LLM Fine-tuning

📅 日期