牛哥精选 · 一年

1

📝 深度技术 arXiv 机器学习 2026-08-11

Matching Accuracy, Different Geometry: Evolution Strategies vs GRPO in LLM Post-Training

进化策略竟能与GRPO精度持平？揭秘两者在LLM后训练中的不同几何路径。

arXiv:2604.01499v2 Announce Type: replace Abstract: Evolution Strategies (ES) have emerged as a scalable gradient-free alternative to reinforcement le…

进化策略 grpo llm后训练强化学习几何性质

2

📝 深度技术 arXiv AI 2026-08-04

Cooperative Coevolution for Resource-Constrained Agentic LLM Post-Training

协同进化替代反向传播，资源受限环境下多轮工具型智能体的全参数后训练更省内存，性能不打折

arXiv:2608.02391v1 Announce Type: new Abstract: Tool-using large language model (LLM) agents produce long, multi-turn trajectories, making gradient-ba…

协同进化进化策略 llm后训练智能体资源高效

3

📝 深度技术 Hacker News LLM 2026-08-02

Show HN: Minimal LLM Post-Training Experiments on an 8GB GPU (SFT, DPO, GRPO)

只需8GB显存，就能跑通SFT、DPO、GRPO全流程，轻松理解DeepSeek-R1背后的推理训练奥秘。

Article URL: https://github.com/pochenai/nano-llm-posttraining Comments URL: https://news.ycombinator.com/item?id=49133851 Points: 6 # Comments: 0

llm后训练 dpo grpo deepseek 小显存

4

📝 深度技术 arXiv 机器学习 2026-07-31

HARGO: Heterogeneity-Aware Reward-Guided Optimization for RL Post-Training of LLMs on HPC Tasks

打破LLM后训练算力瓶颈，异构感知奖励引导优化让HPC任务上的强化学习更高效精准。

arXiv:2607.28301v1 Announce Type: new Abstract: Supervised fine-tuning (SFT) can equip large language models (LLMs) with domain knowledge for high-per…

llm后训练强化学习异构计算 hpc 奖励优化

5

🤖 AI·大模型 arXiv AI 2026-07-14

Reinforcement Learning with Verifiable Physics: Post-training LLMs with Continuous Rewards

巧用物理规则为LLM提供连续奖励信号，让强化学习后训练更可解释、更高效

arXiv:2607.10474v1 Announce Type: cross Abstract: Partial differential equations (PDEs) are foundational to modeling in science and engineering, but c…

强化学习可验证物理连续奖励 llm后训练物理驱动ai

6

📝 深度技术 arXiv 机器学习 2026-06-30

MOPD: Multi-Teacher On-Policy Distillation for Capability Integration in LLM Post-Training

提出多教师在线策略蒸馏框架，高效整合多个大模型能力，优于传统方法

arXiv:2606.30406v1 Announce Type: cross Abstract: Modern large language models (LLMs) rely on reinforcement learning during post-training to push spec…

多教师蒸馏在线策略能力整合 llm后训练强化学习

7

🤖 AI·大模型 arXiv AI 2026-06-24

Tuning without Peeking: Provable Generalization Bounds and Robust LLM Post-Training

提出“无窥视调优”方法，为大模型后训练提供可证明的泛化界限与鲁棒性保障。

arXiv:2507.01752v4 Announce Type: replace-cross Abstract: Gradient-based optimization is the workhorse of deep learning, offering efficient and scalab…

llm 后训练泛化界鲁棒性可证明理论

8

🤖 AI·大模型 arXiv AI 2026-06-23

VRPO: Rethinking Value Modeling for Robust RL under Noisy Supervision in LLM Post-Training

应对LLM后训练中噪声监督，VRPO提出重新思考价值建模实现鲁棒强化学习，值得关注。

arXiv:2508.03058v2 Announce Type: replace-cross Abstract: Reinforcement Learning (RL) in real-world environments often suffers from ambiguous or incom…

vrpo 价值建模强化学习大模型后训练噪声监督

9

📝 深度技术 arXiv AI 2026-06-03

Physics-Guided Policy Optimization with Self-Distillation

物理引导的自我蒸馏策略优化，解决LLM后训练中更新步长信任难题，提升模型对齐效果。

arXiv:2606.03620v1 Announce Type: cross Abstract: Self-distilled policy optimization (SDPO) has become a popular paradigm for LLM post-training, where…

自我蒸馏策略优化 llm后训练物理引导 sdpo

10

📝 深度技术 arXiv AI 2026-06-02

AlphaToken: Decoupling Adaptation and Stability for Path-Aware Response Token Valuation in LLM Post-Training

LLM后训练新突破：AlphaToken通过路径感知机制对响应token进行估值，巧妙解耦适应性与稳定性，提升模型微调效率与鲁棒性。

arXiv:2606.01635v1 Announce Type: cross Abstract: Token selection is pivotal for effective LLM post-training. However, existing methods mostly rely on…

alphatoken llm后训练路径感知 token估值适应性

11

📝 深度技术 arXiv 机器学习 2026-06-01

Consolidating Rewarded Perturbations for LLM Post-Training

把后训练循环从样本空间移到权重空间，用高斯扰动采样+奖励排序集成专家，开辟LLM微调新范式。

arXiv:2605.31494v1 Announce Type: cross Abstract: Post-training of language models is commonly framed as a sample-score-update loop implemented by gra…

llm后训练奖励扰动权重空间高斯采样集成学习

12

📝 深度技术 arXiv 机器学习 2026-05-29

RUBRIC-ARROW: Alternating Pointwise Rubric Reward Modeling for LLM Post-training in Non-verifiable Domains

提出RUBRIC-ARROW方法，通过交替点对点标准奖励建模优化LLM在非可验证领域的后训练性能

arXiv:2605.29156v1 Announce Type: new Abstract: Pointwise reward modeling offers critical signals for LLM post-training, yet struggles with absolute s…

llm后训练奖励模型非可验证领域 rubric-arr 点对点评分

13

📝 深度技术 arXiv AI 2026-05-28

Zipping the Thought: When and How Compressed Reasoning Data Works in LLM Post-Training

探明压缩推理数据在LLM后训练中的生效条件与方式，为效率优化提供理论支撑。

arXiv:2605.28008v1 Announce Type: new Abstract: Large language models (LLMs) can now solve complex problems through long chain-of-thought (CoT) reason…

压缩推理数据 llm后训练推理效率数据压缩模型优化

14

📝 深度技术 arXiv 机器学习 2026-05-23

F-TIS: Harnessing Diverse Models in Collaborative GRPO

GRPO新变体F-TIS：通过多模型协作提升LLM后训练奖励信号多样性，突破单一策略局限。

arXiv:2605.22537v1 Announce Type: new Abstract: Reinforcement learning methods such as GRPO have seen great popularity in LLM post-training. In GRPO, …

grpo 强化学习 llm后训练多模型协作自动回归生成

15

🤖 AI·大模型 arXiv AI 2026-05-23

Unlocking Proactivity in Task-Oriented Dialogue

让保守的LLM在任务对话中主动出击，论文用奖励塑造RL解锁销售场景的前瞻策略。

arXiv:2605.22240v1 Announce Type: new Abstract: Proactive task-oriented dialogue (TOD), such as outbound sales, demands a persuasive agent that active…

任务导向对话主动性说服性智能体 llm后训练强化学习

16

📝 深度技术 arXiv AI 2026-05-20

Beyond GRPO and On-Policy Distillation: An Empirical Sparse-to-Dense Reward Principle for Language-Model Post-Training

提出稀疏到稠密奖励原则，四阶段后训练流程更高效利用稀缺标注数据，为LLM推理优化提供新范式。

arXiv:2605.12483v2 Announce Type: replace-cross Abstract: When labeled verifiable training data is scarce, each checked example should be used where i…

llm后训练强化学习奖励设计 grpo 知识蒸馏

17

📝 深度技术 arXiv NLP 2026-05-19

When Importance Sampling Misallocates Credit: Asymmetric Ratios for Outcome-Supervised RL

最新研究揭示重要性采样在结果监督RL中的信用分配偏差，提出不对称比率改进方法，为LLM后训练提供新视角。

arXiv:2510.06062v2 Announce Type: replace Abstract: Reinforcement learning (RL) has shown great promise in large language models (LLMs) post-training,…

重要性采样结果监督rl grpo 不对称比率 llm后训练

🐂 牛哥精选