1
Debate Training Reduces Reward Hacking in RLAIF
用辩论训练对抗奖励黑客，为AI对齐提供新思路，值得关注。
arXiv:2608.17776v1 Announce Type: new Abstract: We demonstrate that RL finetuning an LLM using debate, a two-player adversarial game between a generat…
2
Voting with the Graph: Stable RLAIF via Topological Consistency Maximization
用图投票和拓扑一致性最大化让RLAIF训练更稳定，清华等团队提出新对齐方法。
arXiv:2510.15514v3 Announce Type: replace Abstract: Reinforcement Learning from AI Feedback (RLAIF) relies on LLM judges as preference measurement ins…
3
ODRPO: Ordinal Decompositions of Discrete Rewards for Robust Policy Optimization
提出序数分解离散奖励的ODRPO方法，提升LLM对齐中策略优化的鲁棒性，直面自动评分器的随机挑战。
arXiv:2605.12667v2 Announce Type: replace-cross Abstract: The alignment of Large Language Models (LLMs) utilizes Reinforcement Learning from AI Feedba…