1
华为开源盘古 openPangu-2.0 的预训练、SFT 代码和后训练 RL 代码正式开源上线
IT之家 9 月 28 日消息，华为今日宣布， 开源盘古 openPangu-2.0 的预训练、 SFT 代码和后训练 RL 代码正式开源上线 。 开源盘古 openPangu 是华为开源 AI 模型品牌，致力于通过昇腾原生训练与推理技术，为业界用好昇腾提供最佳实践参考。 openPangu-2.0…
2
曾任职 OpenAI 的 Anthropic 预训练研究员离职并警告：两家公司正拿人类生命赌博
IT之家 9 月 9 日消息，据《商业内幕》今天（9 日）午间报道，曾在 OpenAI 工作的 Anthropic 研究员雅各布 · 考克森宣布辞职。过去 3 年，他先后在 OpenAI 和 Anthropic 两家 AI 巨头从事预训练研究。 考克森发出警告称：“两家公司的做法都不负责任。它们争相…
3
AI 竞赛升温：曝 OpenAI 已预训练 Bel 模型：超 10T 参数，冲击通用人工智能
OpenAI新模型Bel曝光，超10万亿参数直指AGI，AI竞赛再升级。
IT之家 8 月 26 日消息，消息源 @synthwavedd 今天（8 月 26 日）在 X 平台发布推文，爆料称 OpenAI 已完成预训练下一代模型， 内部代号为 Bel，是 Doug 模型的直接继任者，总参数量超过 10T。 消息称 OpenAI 希望将 Bel 模型打造成为 GPT-6 …
4
What AstroPT knows about galaxies, and what that can teach us about LLMs
从星系到语言模型，AstroPT揭示了预训练模型如何习得结构知识，为LLM研究提供跨学科洞见。
arXiv:2608.22614v1 Announce Type: new Abstract: Interpretability research increasingly asks when concepts emerge during training and whether linear pr…
5
6
设计无边界！中坚科技旗下子公司桦之坚携巨型概念机器人ZERO惊艳2026世界机器人大会
巨型概念人形机器人ZERO惊艳亮相，基于NVIDIA Isaac平台预训练，开启未来机器人新想象。
中坚科技（002779.SZ）旗下桦之坚携概念人形机器人ZERO亮相2026世界机器人大会。
7
Teaching a local LLM to reason about a new domain through continued pretraining
用继续预训练给本地小模型注入新领域知识，实战示例清晰，适合想深入LLM调优的读者。
Article URL: https://www.teachmecoolstuff.com/viewarticle/teaching-a-local-llm-a-new-domain Comments URL: https://news.ycombinator.com/item?id=4938012…
8
How Do Large Language Models Learn Concepts During Continual Pre-Training?
揭秘大模型在持续预训练中如何习得、保留与遗忘概念，理解AI认知的关键研究。
arXiv:2601.03570v2 Announce Type: replace Abstract: Human beings primarily understand the world through concepts (e.g., dog), abstract mental represen…
9
Invariant Pretraining for Robust Code Representations
面向代码智能的预训练新范式，用不变学习增强代码表示的鲁棒性与泛化能力，值得关注。
arXiv:2608.15412v1 Announce Type: cross Abstract: Encoder-based code representation models remain widely deployed for discriminative tasks such as clo…
10
Scaling Domain Data Repetition in LLM Pretraining
LLM预训练中领域数据重复的缩放规律，揭示数据重复如何影响模型性能与训练效率的边界效应。
arXiv:2608.14071v1 Announce Type: new Abstract: As large language models scale, their training-token budgets must also increase to maintain an appropr…
11
TELLME: Test-Enhanced Learning for Language Model Enrichment
用测试增强学习替代海量语料，为语言模型持续预训练省下高额计算成本，稳准快。
arXiv:2608.11788v1 Announce Type: cross Abstract: Continual pre-training (CPT) has been widely adopted as a method for domain adaptation in large lang…
12
Benchmarking Trustworthiness of SLMs: Pre-trained vs. Compressed
预训练与压缩小模型谁更可信？IJCNN权威基准测试给出对比答案。
arXiv:2608.11981v1 Announce Type: new Abstract: Small Language Models (SLMs) have emerged as a more efficient alternative to traditional Large Languag…
13
Diffract: Spectral View of LLM Domain Adaptation
用谱视角剖析大模型领域适应，Diffract方法为持续预训练提供新洞察。
arXiv:2608.10850v1 Announce Type: new Abstract: We study continual pre-training (CPT) as a mechanism for adapting general-purpose large language model…
14
Robustness of transferability estimation metrics for medical imaging
医疗影像中选预训练模型总靠试错？本文系统检验了迁移性估计指标在医学场景下的鲁棒性，帮你少走弯路。
arXiv:2608.09999v1 Announce Type: cross Abstract: In transfer learning, the choice of source model largely influences the performance on a target data…
15
Reducing Pretraining-Generation Mismatch in Diffusion Language Models
直击扩散语言模型预训练与生成阶段的不匹配痛点，提出改进方法，值得算法研究者细读。
arXiv:2608.09424v1 Announce Type: new Abstract: Autoregressive language models align training and use: generation conditions on a clean prompt, and tr…
16
Instability of LLM Pre-Pretraining: It Doesn't Always Help. An Investigation on Multiple Languages
多语言预训练并非总是有益，这项研究揭示了LLM预训练不稳定的关键发现。
arXiv:2608.08800v1 Announce Type: new Abstract: Pretraining LLMs on artificial languages ("pre-pretraining") is a technique that could reportedly incr…
17
One Leak Away: How Pretrained Model Exposure Amplifies Jailbreak Risks in Finetuned LLMs
预训练模型暴露如何放大微调大模型的越狱风险，安全研究必读。
arXiv:2512.14751v3 Announce Type: replace-cross Abstract: Finetuning pretrained large language models (LLMs) has become the standard paradigm for deve…
18
MoCA: Multi-modal Cross-masked Autoencoder for Digital Health Measurements
把多模态健康数据与自编码器结合，交叉掩码预训练让数字医疗测量更精准，值得算法研究者一读
arXiv:2506.02260v4 Announce Type: replace-cross Abstract: Wearable devices enable continuous multi-modal physiological and behavioral monitoring, yet …
19
Speech LLMs in Low-Resource Scenarios: Data Volume Requirements and the Impact of Pretraining on High-Resource Languages
低资源场景下语音大模型的数据需求，以及高资源语言预训练的影响，实验数据扎实，值得一读。
arXiv:2508.05149v2 Announce Type: replace-cross Abstract: Large language models (LLMs) have demonstrated potential in handling spoken inputs for high-…
20
Beyond Initialization Loss: A Systematic Study of Token Embedding Initialization Strategies for LLM Vocabulary Extension
系统评估20余种初始化策略，为LLM词汇扩展提供实用指南。
arXiv:2608.03494v1 Announce Type: cross Abstract: Vocabulary extension is an efficient way to adapt pretrained large language models (LLMs) to new lan…