用MUD搭建AI评估场，揭露LLM裁判在聚合kappa指标下隐藏的失真，挑战传统评估框架的盲区。
Article URL: https://www.lesswrong.com/posts/GPbWyHgx9hCLMdAjc/mud-as-ai-evaluation-and-llm-judge-distortion-in-ways Comments URL: https://news.ycombi…
用机械可解释性方法解剖LLM作为评判者时的内在偏见，揭示不公平机制根源
arXiv:2607.11871v1 Announce Type: cross Abstract: Existing studies of LLM-as-judge scoring bias work predominantly at the input-output level: they per…
一个开源免费工具，帮你用人工标注和LLM裁判双维度评估AI Agent输出质量。
Article URL: https://github.com/AntoineF23/verdict Comments URL: https://news.ycombinator.com/item?id=48875259 Points: 1 # Comments: 0
自博弈评判者打分的是“像不像”而非“对不对”，参考无关LLM裁判存在结构性缺陷。
arXiv:2607.05904v1 Announce Type: new Abstract: Training a language model against its own reference-free judgments (the premise of self-rewarding, sel…
用确定性代码与LLM评判双管齐下，让生成式AI产出可靠的高质量文档。
Article URL: https://www.scala-lang.org/blog/2026/07/06/quality-from-genai.html Comments URL: https://news.ycombinator.com/item?id=48816619 Points: 2 …
新方法RoPoLL让LLM评判者更鲁棒，18倍参数优势却更精准，有效对抗偏见污染。
arXiv:2606.30931v1 Announce Type: new Abstract: The LLM Jury, a Panel of LLM Evaluators (PoLL) reporting consensus scores, has become a practical alte…
新研究揭示LLM裁判在代理任务中打分不可靠，瓶颈与改进思路值得关注
arXiv:2606.29920v1 Announce Type: new Abstract: Rubric-based scoring has become a widely used paradigm in model evaluation, typically with LLM-as-a-Ju…
两大AI评委给Agent回答高分0.85，但Agent从未打开文件，GroundEval揭穿这巨大评分偏差。
Article URL: https://tenureai.dev/writing/llm-as-judge-became-the-default-for-agent-evaluation/ Comments URL: https://news.ycombinator.com/item?id=486…
多模态大语言模型充当评判者是否可靠？本文揭示其应对对抗攻击时的脆弱性并提出提升鲁棒性的方向。
arXiv:2606.15608v1 Announce Type: new Abstract: Multimodal Large Language Models (MLLMs) are increasingly used as automated judges, e.g., for image qu…
揭示LLM作为评判者在QA评估中因知识驱动而失败的深层原因，实验严谨发现惊人。
arXiv:2601.07506v2 Announce Type: replace Abstract: While large language models (LLMs) are increasingly used as automatic judges for question answerin…
揭秘温度参数对LLM评判公正性的关键影响，论文论证了设置温度的必要性。
arXiv:2603.28304v2 Announce Type: replace Abstract: Using large language models (LLMs) as judges for evaluating model outputs has emerged as an import…
Agentic评估 vs LLM评判：在成本、时间和质量之间如何权衡？看这篇讨论。
Comments URL: https://news.ycombinator.com/item?id=48144995 Points: 1 # Comments: 0