首个公开评估Claude Fable 5在生物医学挑战问题的表现，直面基准饱和与评估方法可靠性问题。
arXiv:2607.10849v1 Announce Type: new Abstract: Frontier language models are increasingly evaluated on biomedical benchmarks, but two problems undermi…
多模态大模型视觉计数能力新基准，HoloCount全面评测模型在复杂场景下的计数表现，揭示当前MLLMs的瓶颈。
arXiv:2607.06420v1 Announce Type: new Abstract: Visual counting is a fundamental pillar of multimodal intelligence, requiring a seamless integration o…
被ICML 2026接收，提出用确定性真实值评估大模型长文本生成的不确定性，为LLM可靠性研究提供新工具。
arXiv:2607.03870v1 Announce Type: new Abstract: As LLMs generate increasingly long outputs, effective uncertainty estimation must identify errors at f…
论文揭示多步LLM代理中控制框架导致的信念分歧，拷问基准测试的隐藏偏差。
arXiv:2607.04528v1 Announce Type: new Abstract: Software-agent benchmarks usually report whether an agent solves a task, but the agent reaches that ou…
用逻辑网格谜题精准探测大模型推理中的隐性偏见，一场理性与偏好的博弈实验
arXiv:2511.06160v2 Announce Type: replace Abstract: While recent safety guardrails effectively suppress overtly biased outputs, subtler forms of socia…
揭露LLM评估的致命缺陷：简单平均分数掩盖真实性能差异，这篇论文用数据证明为何必须抛弃当前主流评测方式。
Article URL: https://arxiv.org/abs/2605.11205 Comments URL: https://news.ycombinator.com/item?id=48744350 Points: 1 # Comments: 0
新研究揭示LLM裁判在代理任务中打分不可靠，瓶颈与改进思路值得关注
arXiv:2606.29920v1 Announce Type: new Abstract: Rubric-based scoring has become a widely used paradigm in model evaluation, typically with LLM-as-a-Ju…
用真实购买结果衡量LLM用户模拟器的决策保真度，为对话AI评估提供新标尺
arXiv:2606.20708v1 Announce Type: new Abstract: LLM-as-user-simulation has become core infrastructure for conversational AI: agent benchmarks (tau-ben…
用自然语言推理约束验证，解决LLM个性化评估中表面匹配和计算成本问题，新范式更可解释。
arXiv:2606.16368v1 Announce Type: cross Abstract: Current evaluation paradigms for Large Language Model (LLM) personalization rely heavily on brittle …
LLM评判模型输出的可靠性如同抛硬币？这篇研究系统揭示评估过程中的偏见与随机性问题。
arXiv:2606.13685v1 Announce Type: cross Abstract: LLM-as-a-Judge is now widely used to rank model outputs, train reward models, and populate public le…
LLM多元性新评估方法：通过潜在视角分析模型多样性的前沿研究。
arXiv:2606.13254v1 Announce Type: new Abstract: The growing need to represent diverse perspectives has increased interest in pluralistic LLM generatio…
LLM心理测量评估新视角：揭示自我报告预测行为的条件与原因，为模型行为理解提供理论支撑。
arXiv:2606.12730v1 Announce Type: new Abstract: Anticipating LLM behavioral tendencies from low-cost psychometric probes is critical for safe deployme…
ACL 2026收录论文：系统提出语言模型在开放任务中创造力的自动化评估方法，填补量化空白。
arXiv:2606.11762v1 Announce Type: cross Abstract: Large language models (LLMs) have achieved remarkable progress in language understanding, reasoning,…
深入剖析大语言模型的几何度量方法，揭示其测量原理与适用场景，为AI评估提供新视角
arXiv:2509.25359v2 Announce Type: replace-cross Abstract: We present a systematic stress-test of geometric metrics for LLM evaluation. Rank-based geom…
研究揭示大语言模型在道德推理上的不足，为安全AI发展敲响警钟。
arXiv:2606.11635v1 Announce Type: cross Abstract: For highly capable AI systems to operate safely in dynamic, open-ended environments, they must be ab…
研究者将LLM作为判别器评估合成表格数据，揭示“以假乱真”的潜在风险。
arXiv:2606.09865v1 Announce Type: new Abstract: Privacy and data sharing are often in tension. Many organizations use synthetic data to reduce privacy…
从单纯通过率到多语言执行评估，这篇论文重新定义了开源代码大模型的性能比较标准。
arXiv:2606.08840v1 Announce Type: new Abstract: Code generation models are typically compared using compact execution benchmarks and aggregate pass ra…
挑战无监督单词发现中传统评价指标，提出新视角下的词典评估方法。
arXiv:2606.06183v1 Announce Type: cross Abstract: Building a lexicon from discovered word-like units is a central goal in zero-resource speech process…
论文揭示多智能体LLM系统拓扑结构如何引发记忆泄漏，并提出MAMA评估框架量化风险。
arXiv:2512.04668v4 Announce Type: replace-cross Abstract: Graph topology is a fundamental determinant of memory leakage in multi-agent LLM systems, ye…
大语言模型知识基准新突破：KINA跨越261门学科，解决排名不稳定与标注惰性。
arXiv:2606.05104v1 Announce Type: new Abstract: Knowledge benchmarks for LLMs face three issues: scaling-driven designs that do not operationalize dis…