从零训练113M参数地震大模型，开源全流程管线，自建数据集+分布式训练，值得参考复现。
Article URL: https://github.com/jiazhe868/nanogpt-seis Comments URL: https://news.ycombinator.com/item?id=48885236 Points: 9 # Comments: 2
提出GIFT方法，利用梯度几何信息实现低精度通信，在不牺牲模型精度的前提下显著降低LLM预训练通信开销。
arXiv:2607.07494v1 Announce Type: cross Abstract: Gradient communication is a primary scaling bottleneck in large language model (LLM) pretraining. Co…
联邦学习打破带宽瓶颈，让大模型预训练从数据中心走向协作分布式场景
arXiv:2411.02908v2 Announce Type: replace Abstract: Scaling large language models (LLMs) demands extensive data and computing resources, which are tra…
Piper 让你用少量注解自定义分布式训练策略，真正实现策略与运行时的解耦。
arXiv:2606.11169v1 Announce Type: cross Abstract: Large-scale model training increasingly relies on composing multiple parallelism strategies, such as…
这篇论文提出统一本地通信与更新策略，旨在提升大模型预训练的通信效率，分布式训练的新视角。
arXiv:2606.11081v1 Announce Type: cross Abstract: Communication-efficient pre-training of LLMs is increasingly important as training draws on compute …
大模型训练新突破：灵活上下文并行策略，高效扩展LLM训练边界
arXiv:2602.21788v2 Announce Type: replace-cross Abstract: Scaling long-context capabilities is crucial for Large Language Models (LLMs). However, real…
针对网络故障导致AllReduce集体通信降速问题提出解决方案，助力大规模分布式训练。
arXiv:2606.01680v1 Announce Type: cross Abstract: Network failures are among the most frequent hardware faults in large-scale GPU clusters and a leadi…
探析分布式训练是否挑战AI算力治理，为ICML 2026 TAIGR工作坊论文。
arXiv:2605.29359v1 Announce Type: cross Abstract: Compute governance proposals often rely on the assumption that frontier AI training requires large, …
真实生产环境下的LLM预训练运维经验，504块GPU集群从故障检测到恢复的实证分析。
arXiv:2605.09370v2 Announce Type: replace-cross Abstract: Large-scale AI training is now fundamentally a distributed systems problem, and hardware fai…
突破多模态LLM训练中计算与内存的帕累托前沿，BigMac提出高效优化方案。
arXiv:2605.25451v1 Announce Type: new Abstract: Training multimodal large language models (MLLMs) is challenged by both model and data heterogeneity. …
新论文提出ReCoVer系统，用容错集合和灵活工作负载增强LLM预训练弹性，减少训练中断损失。
arXiv:2605.11215v2 Announce Type: replace-cross Abstract: Pre-training large language models on massive GPU clusters has made hardware faults routine …
提出CRAFT方法解决联邦学习中客户端模型更新冲突，通过冲突消解聚合提升训练效率与模型质量。
arXiv:2605.21317v1 Announce Type: new Abstract: The aggregation of conflicting client updates remains a fundamental bottleneck in federated learning (…
分布式训练新突破：LOSCAR-SGD通过通信计算重叠和延迟校正，实现稀疏模型平均高效加速。
arXiv:2605.20866v1 Announce Type: new Abstract: Communication is a major bottleneck in distributed learning, especially in large-scale settings and in…
新方法MTraining通过分布式动态稀疏注意力，大幅降低超长上下文训练的计算开销。
arXiv:2510.18830v2 Announce Type: replace-cross Abstract: The adoption of long context windows has become a standard feature in Large Language Models …
二阶优化方法加速LLM训练的瓶颈被Asteria运行时系统破解，大幅提升训练效率。
arXiv:2605.16184v1 Announce Type: cross Abstract: Second-order methods offer an attractive path toward more sample-efficient LLM training, but their p…
分布式学习中数据归因的脆弱性：单个参与者可操纵归因值大幅膨胀，挑战定价与审计可信度。
arXiv:2605.15520v1 Announce Type: cross Abstract: Data attribution has become an important component of pricing, auditing, and governance in machine l…
用少量GPU忠实模拟千卡级LLM训练环境，降低开发调试成本与复杂度。
arXiv:2605.15617v1 Announce Type: cross Abstract: Large language model (LLM) training today runs on clusters spanning thousands of GPUs. While this sc…