数据选择新思路：用DPO偏好优化为模型量身筛选训练数据，提升后训练效果。
arXiv:2608.16926v1 Announce Type: new Abstract: Data selection in supervised fine-tuning aims to select a small set of effective samples from large-sc…
只需8GB显存，就能跑通SFT、DPO、GRPO全流程，轻松理解DeepSeek-R1背后的推理训练奥秘。
Article URL: https://github.com/pochenai/nano-llm-posttraining Comments URL: https://news.ycombinator.com/item?id=49133851 Points: 6 # Comments: 0
一文对比RLHF与DPO两种主流大模型训练方法的核心差异与适用场景
In this article, we will look at how that learning actually happens, starting with why instruction-following alone falls short, then walking through t…
对比DPO与RLHF的对齐代价，揭示大模型隐藏的哲学回答偏差
Ask yourself one question. When you talk to ChatGPT or Claude, do you feel like you talk to something that thinks — or something that agrees with you …
反复后训练并非自我提升？新研究揭示DPO持续训练中的“科学遗忘”现象，值得关注。
arXiv:2606.21089v1 Announce Type: new Abstract: Industrial LLM teams often ship behavior updates by repeatedly DPO-training a base model on sequences …
大模型能否自查伦理偏差？新研究引入“良心步骤”用DPO训练实现自我对齐与修正。
arXiv:2606.19527v1 Announce Type: new Abstract: Can Large Language Models (LLMs) discern when their own outputs are misaligned with human ethics? And …
为LLM后训练设计新方法REDIPO，在不损失对齐的前提下恢复输出多样性，解决指令回答单一化问题。
arXiv:2605.30021v2 Announce Type: replace Abstract: Many open-ended instructions have multiple valid answers that users can benefit from seeing, but p…
物理引导的自我蒸馏策略优化，解决LLM后训练中更新步长信任难题，提升模型对齐效果。
arXiv:2606.03620v1 Announce Type: cross Abstract: Self-distilled policy optimization (SDPO) has become a popular paradigm for LLM post-training, where…
DPO统一范式Uni-DPO，动态优化LLM偏好，解决数据质量差异问题。
arXiv:2506.10054v4 Announce Type: replace-cross Abstract: Direct Preference Optimization (DPO) has emerged as a cornerstone of reinforcement learning …
4行代码实现应用与代理间实时通信，基于MQTT低延迟无轮询，定价透明。
CloudPostOffice is realtime messaging for apps, scripts, and AI agents. No MQTT broker setup or configuration, no infrastructure to manage. p1 = cpo.p…
单GPU实现凸优化方法，高效解决LLM偏好对齐难题，降低RLHF计算成本。
arXiv:2605.23244v1 Announce Type: new Abstract: Fine-tuning large language models (LLMs) to align with human preferences has driven the success of sys…
多LLM协同训练新框架，自适应参考演化提升偏好优化效果。
arXiv:2602.02709v3 Announce Type: replace Abstract: Recent multi-LLM agent systems have shown promising capabilities for automated problem-solving, ye…
一篇系统梳理LLM后训练中强化学习的综述，涵盖RLHF、DPO、RLVR等前沿方法
arXiv:2407.16216v4 Announce Type: replace Abstract: Large language models (LLMs) trained via pretraining and supervised fine-tuning (SFT) can still pr…
新方法用DPO隐式奖励差距衡量样本难度，自动筛选高质量偏好数据，提升模型训练效率。
arXiv:2508.04149v2 Announce Type: replace-cross Abstract: Aligning large language models (LLMs) with human preferences is a critical challenge in AI r…
提出TokenRatio方法，通过比值匹配实现原则性token级偏好优化，突破DPO的序列级局限，更精准对齐语言模型
arXiv:2605.12288v2 Announce Type: replace-cross Abstract: Direct Preference Optimization (DPO) is a widely used RL-free method for aligning language m…
揭示DPO与PPO本质差异，挑战“监督学习vs强化学习”传统认知的深度技术论文。
arXiv:2512.00778v2 Announce Type: replace Abstract: Preference optimization (PO) is indispensable for large language models (LLMs), with methods such …