SFT Overtraining Predicts Rank Inversion via Entropy Collapse Under RLVR
ICML 2026 研究揭示:监督微调过度训练会通过熵塌陷导致强化学习下模型输出排名反转,对齐策略需要警惕。
arXiv:2606.18487v1 Announce Type: new Abstract: The standard heuristic of selecting the SFT checkpoint with the highest pass@1 for GRPO can fail when …