1
How LLMs Learn to Be Helpful (RLHF vs DPO)
一文对比RLHF与DPO两种主流大模型训练方法的核心差异与适用场景
In this article, we will look at how that learning actually happens, starting with why instruction-following alone falls short, then walking through t…
2
When Implausible Tokens Get Reinforced: Tail-Aware Credit Calibration for LLM Reinforcement Learning
解决LLM强化学习中不合理token被错误放大的问题，提出尾部感知信用校准新方法。
arXiv:2607.07976v1 Announce Type: cross Abstract: Reinforcement learning (RL) has achieved remarkable success in enhancing the reasoning capabilities …
3
DPO vs RLHF: The Alignment Tax You Pay Without Knowing
对比DPO与RLHF的对齐代价，揭示大模型隐藏的哲学回答偏差
Ask yourself one question. When you talk to ChatGPT or Claude, do you feel like you talk to something that thinks — or something that agrees with you …
4
What are Key Factors for Updates in RL for LLM Reasoning?
一篇剖析强化学习更新大模型推理能力的关键因素研究，揭示影响性能的核心变量与训练策略。
arXiv:2606.22570v1 Announce Type: new Abstract: Reinforcement Learning from Verifiable Rewards (RLVR) has emerged as a promising framework for enhanci…
5
Certifiable Safe RLHF: Semantic Grounding and Fixed Penalty Constraint Optimization for Safer LLM Alignment
语义基础+固定惩罚约束优化，让大模型对齐过程获得可认证的安全保障
arXiv:2510.03520v2 Announce Type: replace-cross Abstract: Ensuring safety is a foundational requirement for large language models (LLMs). Achieving an…
6
Beyond Uniform Token-Level Trust Region in LLM Reinforcement Learning
非均匀令牌级信任区域优化，突破传统限制提升大模型强化学习训练稳定性。
arXiv:2606.10968v1 Announce Type: cross Abstract: Reinforcement learning with verifiable rewards (RLVR) has become standard for improving LLM reasonin…
7
A Unifying Lens on Reward Uncertainty in RLHF
从统一视角剖析RLHF中的奖励不确定性，为强化学习与人类反馈对齐提供新见解
arXiv:2606.09073v1 Announce Type: new Abstract: Reinforcement learning from human feedback (RLHF) is bottlenecked by \emph{reward hacking}, where the …
8
What Do People Actually Want From AI? Mapping Preference Plurality
顶会论文揭示RLHF聚合偏好的根本缺陷，系统绘制人类对AI的真实多元需求图谱
arXiv:2606.06674v1 Announce Type: new Abstract: Large Language Models (LLMs) are often fine-tuned through Reinforcement Learning from Human Feedback (…
9
ActiveUltraFeedback: Efficient Preference Data Generation using Active Learning
用主动学习策略精准筛选高价值偏好数据，大幅降低RLHF数据标注成本，大模型偏好对齐的新效率方案。
arXiv:2603.09692v2 Announce Type: replace Abstract: Reinforcement Learning from Human Feedback (RLHF) has become the standard for aligning Large Langu…
10
Isolating LLM Lexical Bias: A Curation-Free Triangulated Metric for Preference-Stage Learning
提出一种无需数据整理的三角测量指标，精准隔离LLM在偏好学习阶段的词汇偏差。
arXiv:2606.00334v1 Announce Type: cross Abstract: Various language domains have undergone remarkable changes in recent years; these shifts are largely…
11
Uni-DPO: A Unified Paradigm for Dynamic Preference Optimization of LLMs
DPO统一范式Uni-DPO，动态优化LLM偏好，解决数据质量差异问题。
arXiv:2506.10054v4 Announce Type: replace-cross Abstract: Direct Preference Optimization (DPO) has emerged as a cornerstone of reinforcement learning …
12
Leveraging Verifier-Based Reinforcement Learning in Image Editing
将RLHF引入图像编辑的新范式，提出基于验证器的强化学习解决奖励模型缺失瓶颈。
arXiv:2604.27505v2 Announce Type: replace Abstract: While Reinforcement Learning from Human Feedback (RLHF) has become a pivotal paradigm for text-to-…
13
Reinforcement Learning for LLM Post-Training: A Survey
一篇系统梳理LLM后训练中强化学习的综述，涵盖RLHF、DPO、RLVR等前沿方法
arXiv:2407.16216v4 Announce Type: replace Abstract: Large language models (LLMs) trained via pretraining and supervised fine-tuning (SFT) can still pr…
14
Not Just RLHF: Why Alignment Alone Won't Fix Multi-Agent Sycophancy
研究发现多智能体系统在同伴分歧下“屈服”并非RLHF特有，基础模型同样存在该漏洞，挑战了传统对齐认知。
arXiv:2605.12991v2 Announce Type: replace Abstract: LLM-based multi-agent pipelines flip from correct to incorrect answers under simulated peer disagr…
15
Beyond RLHF: A Unified Theoretical Framework of Alignment
一份超越RLHF的统一对齐理论框架，抽象形式化多种对齐算法并揭示内在联系，为AI安全提供新视角。
arXiv:2506.01523v2 Announce Type: replace Abstract: Alignment via reinforcement learning from human feedback (RLHF) has become the dominant paradigm f…
16
General Preference Reinforcement Learning
NeurIPS 2026投稿，提出一种通用的偏好强化学习方法，为RLHF等领域提供更坚实的理论基础。
arXiv:2605.18721v1 Announce Type: new Abstract: Post-training has split large language model (LLM) alignment into two largely disconnected tracks. Onl…
17
The Information Theory Behind Why AI Writing Sucks
从信息论看AI写作为何千篇一律，揭开RLHF导致的“注释者共识方言”真相。
Article URL: https://www.pangram.com/blog/joe-stech-information-theory-why-ai-writing-sucks Comments URL: https://news.ycombinator.com/item?id=4819646…
18
Adaptive Layerwise Perturbation: Unifying Off-Policy Corrections for LLM RL
统一离策略修正的自适应逐层扰动方法，为LLM强化学习提供更高效的训练策略。
arXiv:2603.19470v3 Announce Type: replace Abstract: Off-policy problems such as policy staleness and training--inference mismatch have become a major …