揭秘LLM拆分联邦微调中的深度-性能权衡，为分布式训练优化提供新视角。
arXiv:2608.22188v1 Announce Type: new Abstract: Split Federated Fine-tuning (SFF) is a promising paradigm for scaling Large Language Models (LLMs) by …
强化学习训练中混合rollout调度的新突破，跳出前缀局部性束缚，为分布式RL提速提供新思路。
arXiv:2608.11152v1 Announce Type: cross Abstract: Modern reinforcement learning (RL) post-training pipelines for large language models (LLMs) increasi…
大模型预训练遇卡顿不知根源？SCOUT用共识异常检测精准锁定故障节点，摆脱同步困局。
arXiv:2608.11034v1 Announce Type: cross Abstract: In LLM pre-training, synchronization propagates rank-local stalls, slowdowns, and numerical errors i…
用两台 NVIDIA DGX Spark 组远程集群，通过 Tailscale 打通分布式 LLM 训练与网络威胁情报微调，实操部署手册齐全。
arXiv:2608.07226v1 Announce Type: cross Abstract: Compact AI systems make local language-model experimentation increasingly accessible, yet practical …
突破带宽限制，DiLoCo类方法如何平衡通信与性能，提升LLM预训练效率
arXiv:2508.15706v3 Announce Type: replace Abstract: Communication-efficient distributed training algorithms (e.g., DiLoCo) have received considerable …
从零训练113M参数地震大模型，开源全流程管线，自建数据集+分布式训练，值得参考复现。
Article URL: https://github.com/jiazhe868/nanogpt-seis Comments URL: https://news.ycombinator.com/item?id=48885236 Points: 9 # Comments: 2
提出GIFT方法，利用梯度几何信息实现低精度通信，在不牺牲模型精度的前提下显著降低LLM预训练通信开销。
arXiv:2607.07494v1 Announce Type: cross Abstract: Gradient communication is a primary scaling bottleneck in large language model (LLM) pretraining. Co…
用模型归因对抗联邦学习数据异质性，思路新颖且直击分布式训练痛点
arXiv:2606.31742v1 Announce Type: cross Abstract: Explainable AI (XAI) methods have demonstrated significant success in recent years at identifying re…
万亿参数模型训练中，检查点瓶颈如何破？这项研究提出可组合状态提供者的可扩展方案，直击分布式训练痛点。
arXiv:2601.16956v1 Announce Type: cross Abstract: The rapid growth of Large Transformer-based models, specifically Large Language Models (LLMs), now s…
联邦MoE架构打破全副本部署瓶颈，让分布式模型训练更高效。
arXiv:2606.19025v1 Announce Type: new Abstract: Pre-training Large Language Models (LLMs) typically demands large-scale infrastructure with tightly co…
联邦学习打破带宽瓶颈，让大模型预训练从数据中心走向协作分布式场景
arXiv:2411.02908v2 Announce Type: replace Abstract: Scaling large language models (LLMs) demands extensive data and computing resources, which are tra…
联邦LoRA新方案FedRot-LoRA，创新解决旋转错位问题，提升模型聚合效率与收敛速度。
arXiv:2602.23638v3 Announce Type: replace-cross Abstract: Federated LoRA provides a communication-efficient mechanism for fine-tuning large language m…
Piper 让你用少量注解自定义分布式训练策略，真正实现策略与运行时的解耦。
arXiv:2606.11169v1 Announce Type: cross Abstract: Large-scale model training increasingly relies on composing multiple parallelism strategies, such as…
这篇论文提出统一本地通信与更新策略，旨在提升大模型预训练的通信效率，分布式训练的新视角。
arXiv:2606.11081v1 Announce Type: cross Abstract: Communication-efficient pre-training of LLMs is increasingly important as training draws on compute …
大模型训练新突破：灵活上下文并行策略，高效扩展LLM训练边界
arXiv:2602.21788v2 Announce Type: replace-cross Abstract: Scaling long-context capabilities is crucial for Large Language Models (LLMs). However, real…
针对网络故障导致AllReduce集体通信降速问题提出解决方案，助力大规模分布式训练。
arXiv:2606.01680v1 Announce Type: cross Abstract: Network failures are among the most frequent hardware faults in large-scale GPU clusters and a leadi…
探析分布式训练是否挑战AI算力治理，为ICML 2026 TAIGR工作坊论文。
arXiv:2605.29359v1 Announce Type: cross Abstract: Compute governance proposals often rely on the assumption that frontier AI training requires large, …
真实生产环境下的LLM预训练运维经验，504块GPU集群从故障检测到恢复的实证分析。
arXiv:2605.09370v2 Announce Type: replace-cross Abstract: Large-scale AI training is now fundamentally a distributed systems problem, and hardware fai…
突破多模态LLM训练中计算与内存的帕累托前沿，BigMac提出高效优化方案。
arXiv:2605.25451v1 Announce Type: new Abstract: Training multimodal large language models (MLLMs) is challenged by both model and data heterogeneity. …
新论文提出ReCoVer系统，用容错集合和灵活工作负载增强LLM预训练弹性，减少训练中断损失。
arXiv:2605.11215v2 Announce Type: replace-cross Abstract: Pre-training large language models on massive GPU clusters has made hardware faults routine …