教你用记忆化指导数据重用的策略，在不大幅延长训练时间的前提下提升LLM训练效率，顶级会议论文的实用思路。
arXiv:2607.04969v1 Announce Type: new Abstract: The training paradigm of large language models has shifted from traditional one-pass training to multi…
多目标强化学习调度数据，让大模型预训练更高效。
arXiv:2606.24133v1 Announce Type: new Abstract: The composition of training data, governed by the diversity of sources and their mixing strategy, is a…
融合反向传播与优化器阶段的梯度处理，大幅降低LLM训练的内存峰值，突破显存瓶颈的新方案。
arXiv:2606.22932v1 Announce Type: new Abstract: Reverse-mode differentiation computes every weight gradient, writes it to memory, and only then lets t…
发现多数视觉指令样本可通过语言模式解决，提出无需训练的数据选择方法提升跨模态学习
arXiv:2603.09715v2 Announce Type: replace Abstract: Visual instruction tuning is crucial for improving vision-language large models (VLLMs). However, …
从数据、内存、计算三方面统一梳理LLM训练效率优化策略，系统总结前沿方法
arXiv:2606.10706v1 Announce Type: cross Abstract: Resource constraints increasingly determine what can be trained, fine-tuned, and deployed in large l…
破解光学储层计算过拟合难题，提出高效训练新原则
arXiv:2606.10130v1 Announce Type: cross Abstract: Reservoir computers benefit from the inherent complexity of optical phenomena, which provide rich, o…
提出在线验证提示选择方法，大幅提升大推理模型RL训练效率
arXiv:2603.25184v2 Announce Type: replace Abstract: Reinforcement learning (RL) has become essential for post-training large language models (LLMs) in…
用从未发生的对话数据高效训练语音识别模型，突破数据瓶颈
arXiv:2606.03957v1 Announce Type: cross Abstract: Conversational ASR for lower-resource languages and niche domains is limited by the scarcity of doma…
提出原则性样本选择方法提升LLM退火效率，向高质量模型训练迈进。
arXiv:2605.31175v1 Announce Type: new Abstract: The annealing phase is a pivotal convergence stage in LLM pre-training that ultimately determines fina…
新型动态有向图约束数据调度方法，大幅提升LLM训练效率与收敛速度。
arXiv:2605.31164v1 Announce Type: cross Abstract: Training data plays a central role in large language models (LLMs) optimization, motivating extensiv…
用离线强化学习高效微调大模型，代码生成性能与训练成本双优化
arXiv:2605.28409v1 Announce Type: new Abstract: Post-training using online reinforcement learning (RL) is an important training step for LLMs, includi…
8张GPU训1万亿参数大模型，Orbit项目降低巨模型训练门槛
See this project https://github.com/Sphere-AI-Lab/orbit. Comments URL: https://news.ycombinator.com/item?id=48305896 Points: 1 # Comments: 0
面向大语言模型的高效差分隐私训练方法，提出随机裁剪机制优化DP-SGD，兼顾隐私与性能。
arXiv:2605.24879v1 Announce Type: new Abstract: Large language models (LLMs) are trained on vast datasets that may contain sensitive information. Diff…
冻结深层、训练浅层，可解释的层分配策略让持续预训练更高效
arXiv:2605.11416v2 Announce Type: replace Abstract: Selective layer-wise updates are essential for low-cost continued pre-training of Large Language M…
颠覆认知？弱教师模型也能有效蒸馏LLM，预训练阶段教师强度并非关键。
arXiv:2605.23857v1 Announce Type: new Abstract: Knowledge distillation generally assumes a strong-to-weak relationship where stronger teachers yield b…
突破传统缩放定律，HRM-Text提出更高效的预训练方法，大幅降低大模型训练成本。
arXiv:2605.20613v1 Announce Type: new Abstract: The current pretraining paradigm for large language models relies on massive compute and internet-scal…
多模态大模型训练新范式：阶段感知稀疏性动态消除冗余，大幅提升效率而保持性能。
arXiv:2509.18150v2 Announce Type: replace Abstract: Multimodal Large Language Models (MLLMs) have demonstrated outstanding performance across a variet…
提出Token叠加技术，颠覆预训练效率瓶颈，大幅降低算力需求，LLM训练优化必读。
arXiv:2605.06546v2 Announce Type: replace Abstract: Pre-training of Large Language Models is often prohibitively expensive and inefficient at scale, r…
提出异构感知数据集调度方法，提升音频大模型训练效率与效果的新方案。
arXiv:2605.19101v1 Announce Type: cross Abstract: Training general-purpose Audio Large Language Models (ALLMs) across diverse datasets is essential fo…
突破大模型长上下文推理瓶颈，百步内将全注意力高效转为稀疏，平衡效率与精度。
arXiv:2605.16928v1 Announce Type: new Abstract: Long-context inference in large language models is bottlenecked by the quadratic cost of full attentio…