1
How LLMs Learn to Be Helpful (RLHF vs DPO)
一文对比RLHF与DPO两种主流大模型训练方法的核心差异与适用场景
In this article, we will look at how that learning actually happens, starting with why instruction-following alone falls short, then walking through t…
2
DPO vs RLHF: The Alignment Tax You Pay Without Knowing
对比DPO与RLHF的对齐代价，揭示大模型隐藏的哲学回答偏差
Ask yourself one question. When you talk to ChatGPT or Claude, do you feel like you talk to something that thinks — or something that agrees with you …
3
Emergent Alignment
大模型能否自查伦理偏差？新研究引入“良心步骤”用DPO训练实现自我对齐与修正。
arXiv:2606.19527v1 Announce Type: new Abstract: Can Large Language Models (LLMs) discern when their own outputs are misaligned with human ethics? And …
4
Recovering Diversity Without Losing Alignment: A DPO Recipe for Post-Trained LLMs
为LLM后训练设计新方法REDIPO，在不损失对齐的前提下恢复输出多样性，解决指令回答单一化问题。
arXiv:2605.30021v2 Announce Type: replace Abstract: Many open-ended instructions have multiple valid answers that users can benefit from seeing, but p…
5
Physics-Guided Policy Optimization with Self-Distillation
物理引导的自我蒸馏策略优化，解决LLM后训练中更新步长信任难题，提升模型对齐效果。
arXiv:2606.03620v1 Announce Type: cross Abstract: Self-distilled policy optimization (SDPO) has become a popular paradigm for LLM post-training, where…
6
Uni-DPO: A Unified Paradigm for Dynamic Preference Optimization of LLMs
DPO统一范式Uni-DPO，动态优化LLM偏好，解决数据质量差异问题。
arXiv:2506.10054v4 Announce Type: replace-cross Abstract: Direct Preference Optimization (DPO) has emerged as a cornerstone of reinforcement learning …
7
Show HN: CloudPostOffice – Send messages between apps and agents in 4 lines
4行代码实现应用与代理间实时通信，基于MQTT低延迟无轮询，定价透明。
CloudPostOffice is realtime messaging for apps, scripts, and AI agents. No MQTT broker setup or configuration, no infrastructure to manage. p1 = cpo.p…
8
Convex Optimization for Alignment and Preference Learning on a Single GPU
单GPU实现凸优化方法，高效解决LLM偏好对齐难题，降低RLHF计算成本。
arXiv:2605.23244v1 Announce Type: new Abstract: Fine-tuning large language models (LLMs) to align with human preferences has driven the success of sys…
9
ATLAS: A Multi-LLM Training Framework for EvoDPO with Adaptive Reference Evolution
多LLM协同训练新框架，自适应参考演化提升偏好优化效果。
arXiv:2602.02709v3 Announce Type: replace Abstract: Recent multi-LLM agent systems have shown promising capabilities for automated problem-solving, ye…
10
Reinforcement Learning for LLM Post-Training: A Survey
一篇系统梳理LLM后训练中强化学习的综述，涵盖RLHF、DPO、RLVR等前沿方法
arXiv:2407.16216v4 Announce Type: replace Abstract: Large language models (LLMs) trained via pretraining and supervised fine-tuning (SFT) can still pr…
11
Difficulty-Based Preference Data Selection by DPO Implicit Reward Gap
新方法用DPO隐式奖励差距衡量样本难度，自动筛选高质量偏好数据，提升模型训练效率。
arXiv:2508.04149v2 Announce Type: replace-cross Abstract: Aligning large language models (LLMs) with human preferences is a critical challenge in AI r…
12
TokenRatio: Principled Token-Level Preference Optimization via Ratio Matching
提出TokenRatio方法，通过比值匹配实现原则性token级偏好优化，突破DPO的序列级局限，更精准对齐语言模型
arXiv:2605.12288v2 Announce Type: replace-cross Abstract: Direct Preference Optimization (DPO) is a widely used RL-free method for aligning language m…
13
What Is Preference Optimization Doing, and Why?
揭示DPO与PPO本质差异，挑战“监督学习vs强化学习”传统认知的深度技术论文。
arXiv:2512.00778v2 Announce Type: replace Abstract: Preference optimization (PO) is indispensable for large language models (LLMs), with methods such …