给大模型强化学习装上“特权价值函数”，让模型更懂自己表现，训练效率有望大幅提升。
arXiv:2608.16739v1 Announce Type: new Abstract: Reinforcement learning algorithms for Large Language Models (LLMs) are largely distinguished by their …
从偏好平均视角揭示RLHF中程序公平性缺陷，为对齐公平研究提供新警示。
arXiv:2608.10126v1 Announce Type: new Abstract: Reinforcement Learning from Human Feedback (RLHF) aggregates heterogeneous preferences into a single r…
跟着作者手搓ChatGPT核心算法，第8天拆解PPO，发现强化学习原来没那么神秘。
SERIES: Learning RL and JAX in Public - from zero to DeepMind :) When people ask "how was ChatGPT trained?", the answer usually involves RLHF - Reinfo…
生成式大模型蒸馏奖励模型新方法，解决偏好标注昂贵难题。
arXiv:2601.14032v2 Announce Type: replace Abstract: Reward models (RMs) play a pivotal role in aligning large language models (LLMs) with human prefer…
提出归一化奖励方法，提升偏好优化训练稳定性与效果
arXiv:2607.16240v1 Announce Type: cross Abstract: Direct Alignment Algorithms (DAAs) such as DPO have become a common way to post-train and align LLMs…
一文对比RLHF与DPO两种主流大模型训练方法的核心差异与适用场景
In this article, we will look at how that learning actually happens, starting with why instruction-following alone falls short, then walking through t…
解决LLM强化学习中不合理token被错误放大的问题，提出尾部感知信用校准新方法。
arXiv:2607.07976v1 Announce Type: cross Abstract: Reinforcement learning (RL) has achieved remarkable success in enhancing the reasoning capabilities …
对比DPO与RLHF的对齐代价，揭示大模型隐藏的哲学回答偏差
Ask yourself one question. When you talk to ChatGPT or Claude, do you feel like you talk to something that thinks — or something that agrees with you …
针对列表级偏好优化提出分布鲁棒方法，提升对齐稳定性与泛化能力。
arXiv:2607.01715v1 Announce Type: new Abstract: Existing robust preference optimization for language-model alignment mainly studies pairwise supervisi…
直击偏好分布偏移痛点，用分布鲁棒优化为RLHF提供更可靠的对齐方案
arXiv:2503.00539v2 Announce Type: replace Abstract: Reinforcement learning from human feedback (RLHF) has evolved to be one of the main methods for fi…
聚焦LLM推理中奖励建模的设计、挑战与评估，为强化学习优化提供系统视角。
arXiv:2602.09305v2 Announce Type: replace Abstract: Large Language Models (LLMs) demonstrate transformative potential, yet their reasoning remains inc…
提出数据依赖探索新策略，提升在线RLHF样本效率，值得算法研究者细读。
arXiv:2605.04477v2 Announce Type: replace Abstract: Online reinforcement learning from human feedback (RLHF) has emerged as a promising paradigm for a…
强化学习推理新思路：不直接给答案，引导模型在rollout中自省纠错，提升推理深度。
arXiv:2510.09388v2 Announce Type: replace Abstract: Reinforcement Learning (RL) has become a key driver for enhancing the long chain-of-thought (CoT) …
揭秘大模型“口是心非”：偏好未化为激励，效用与行为存在隐性鸿沟
arXiv:2606.22974v1 Announce Type: new Abstract: Recent work on preference elicitation in large language models (LLMs) has demonstrated that, when give…
一篇剖析强化学习更新大模型推理能力的关键因素研究，揭示影响性能的核心变量与训练策略。
arXiv:2606.22570v1 Announce Type: new Abstract: Reinforcement Learning from Verifiable Rewards (RLVR) has emerged as a promising framework for enhanci…
应对奖励模型不可靠问题，提出不确定性感知方法，让RLHF更稳定高效
arXiv:2606.19818v1 Announce Type: cross Abstract: Reinforcement learning from human feedback (RLHF) aligns large language models by training reward mo…
语义基础+固定惩罚约束优化，让大模型对齐过程获得可认证的安全保障
arXiv:2510.03520v2 Announce Type: replace-cross Abstract: Ensuring safety is a foundational requirement for large language models (LLMs). Achieving an…
非均匀令牌级信任区域优化，突破传统限制提升大模型强化学习训练稳定性。
arXiv:2606.10968v1 Announce Type: cross Abstract: Reinforcement learning with verifiable rewards (RLVR) has become standard for improving LLM reasonin…
从统一视角剖析RLHF中的奖励不确定性，为强化学习与人类反馈对齐提供新见解
arXiv:2606.09073v1 Announce Type: new Abstract: Reinforcement learning from human feedback (RLHF) is bottlenecked by \emph{reward hacking}, where the …
顶会论文揭示RLHF聚合偏好的根本缺陷，系统绘制人类对AI的真实多元需求图谱
arXiv:2606.06674v1 Announce Type: new Abstract: Large Language Models (LLMs) are often fine-tuned through Reinforcement Learning from Human Feedback (…