1
How LLMs Learn to Be Helpful (RLHF vs DPO)
一文对比RLHF与DPO两种主流大模型训练方法的核心差异与适用场景
In this article, we will look at how that learning actually happens, starting with why instruction-following alone falls short, then walking through t…
2
Enhancing LLMs through human feedback: a journey towards self-improvement
从人类反馈到大模型自我进化，看最新研究成果如何用反馈驱动LLM性能跃升。
arXiv:2607.11267v1 Announce Type: cross Abstract: In the rapidly evolving landscape of information retrieval systems, the ability to adapt and improve…
3
A Unifying Lens on Reward Uncertainty in RLHF
从统一视角剖析RLHF中的奖励不确定性，为强化学习与人类反馈对齐提供新见解
arXiv:2606.09073v1 Announce Type: new Abstract: Reinforcement learning from human feedback (RLHF) is bottlenecked by \emph{reward hacking}, where the …
4
Voting with the Graph: Stable RLAIF via Topological Consistency Maximization
用图投票和拓扑一致性最大化让RLAIF训练更稳定，清华等团队提出新对齐方法。
arXiv:2510.15514v3 Announce Type: replace Abstract: Reinforcement Learning from AI Feedback (RLAIF) relies on LLM judges as preference measurement ins…
5
RLBFF: Binary Flexible Feedback to bridge between Human Feedback & Verifiable Rewards
提出RLBFF，用二元灵活反馈桥接人类偏好与可验证奖励，提升大模型对齐效率。
arXiv:2509.21319v3 Announce Type: replace-cross Abstract: Reinforcement Learning with Human Feedback (RLHF) and Reinforcement Learning with Verifiable…
6
Fine-tuning GPT-2 from human preferences
OpenAI分享用人类反馈微调GPT-2（774M参数）的实践，发现模型学会复制原文来迎合标注者偏好，揭示了偏好对齐中的反直觉现象。
We’ve fine-tuned the 774M parameter GPT-2 language model using human feedback for various tasks, successfully matching the preferences of the external…
7
Our approach to alignment research
OpenAI详解对齐研究策略：提升AI从人类反馈学习，目标打造能自动解决其余对齐难题的超级对齐系统。
We are improving our AI systems’ ability to learn from human feedback and to assist humans at evaluating AI. Our goal is to build a sufficiently align…