提出AtelierEval框架，从智能体视角系统性评估人类与LLM生成文生图提示词的能力，被ICML 2026录用，带来全新评价维度。
arXiv:2605.22645v1 Announce Type: new Abstract: Text-to-image (T2I) systems increasingly rely on upstream prompters, either humans or multimodal large…
首个专门评估LLM集成知识图谱生成的基准测试发布，填补该领域关键空白。
arXiv:2605.19518v1 Announce Type: new Abstract: Generating Knowledge Graphs (KGs) remains one of the most time-consuming and labor-intensive tasks for…
提出新指标RI，衡量大模型在事实性任务中拒绝未知问题的能力，提升可靠性。
arXiv:2510.01782v2 Announce Type: replace Abstract: Large Language Models (LLMs) should refuse to answer questions beyond their knowledge. This capabi…
质疑LLM在科研评估中的可靠性，这项研究揭示了关键局限。
arXiv:2605.19196v1 Announce Type: new Abstract: Deep research agents increasingly automate complex information-seeking tasks, producing evidence-groun…
超越静态基准，探索LLM评估新范式，引入游戏化思想打破传统测试局限。
arXiv:2605.19377v1 Announce Type: new Abstract: As jailbreaks, adversarially crafted inputs that bypass safety constraints, continue to be discovered …
数学家精选的Soohak基准测试，专攻LLM科研级数学推理能力，挑战最高阶思维极限
arXiv:2605.09063v2 Announce Type: replace Abstract: Following the recent achievement of gold-medal performance on the IMO by frontier LLMs, the commun…
研究如何用LLM代理模拟退休态度，创新性结合人口统计与调查锚点，启发AI社会科学应用。
arXiv:2605.16303v1 Announce Type: cross Abstract: Large language models (LLM) agents may offer tools to predict human responses to surveys. A common t…
探讨如何用LLM评估人效，量化所需人类评审数量，高效平衡AI系统评估的成本与质量。
arXiv:2605.16354v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly used as automated evaluators of AI systems, including in…
新基准GIM通过整合多个认知领域评估大模型，避免死记硬背与脱离现实的推理。
arXiv:2605.18663v1 Announce Type: cross Abstract: As LLM benchmarks saturate, the evaluation community has pursued two strategies to increase difficul…
开源项目LLM INQUISITOR提供真实场景下的LLM行为评估框架，专注长任务和实用性，而非基准测试。
Article URL: https://github.com/AssimilatedHuman/LLM-Inquisitor Comments URL: https://news.ycombinator.com/item?id=48207330 Points: 1 # Comments: 0
基于图的评估框架，让LLM领域评估更全面、抗污染、可维护
arXiv:2508.20810v3 Announce Type: replace Abstract: Rigorous evaluation of domain-specific language models requires benchmarks that are comprehensive,…
Spotify分享如何用LLM评估提升A/B实验命中率，将测试漏斗与实验反馈循环结合，降低无效成本。
TL;DR LLM evals, automated judges that assess relevance, coherence, and quality at scale, are a powerful new... The post Better Experiments with LLM E…
针对静态基准测试的局限，推出MathArena平台，用动态、可扩展的评估助力LLM数学能力衡量
arXiv:2605.00674v2 Announce Type: replace Abstract: Large language models (LLMs) are becoming increasingly capable mathematical collaborators, but sta…
首个基于潜在歧义理论构建的中文歧义理解评估数据集，半自动管道生成，规模可扩展
arXiv:2605.15635v1 Announce Type: new Abstract: Linguistic ambiguity is critical to the robustness of Large Language Models (LLMs), yet existing resea…
自监督双轨框架首次解耦结构与内容，精准评估LLM在Web系统中的表现。
arXiv:2601.19923v2 Announce Type: replace-cross Abstract: As Large Language Models (LLMs) evolve into the core of Web-based autonomous agents and comp…
提出轨迹级评估框架LEAP，首次量化LLM在科学设计中的迭代学习过程，而非仅关注结果快照。
arXiv:2605.15341v1 Announce Type: cross Abstract: LLMs are increasingly deployed in autonomous laboratories, under the assumption that their domain pr…
探讨地缘政治场景下大语言模型安全评估的创新方法，助力国家安全与公共安全防护。
arXiv:2605.14152v1 Announce Type: cross Abstract: Safety evaluations for large language models (LLMs) increasingly target high-stakes National Securit…
面向金融领域的分层LLM评估基准，覆盖专业知识与技术分析，补足现有基准缺口。
arXiv:2605.15482v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly being applied to financial analysis, reporting, investme…
直达前沿AI记忆系统研究论文，评估LLM在多社交群组中的表现，洞察AI记忆适配瓶颈
arXiv:2605.17789v1 Announce Type: new Abstract: Memory systems for AI assistants were built for single-user dialogue and fail characteristically when …
一键诊断LLM评分格式偏差，定位注意力路径中的决策缺陷，让模型评判更可靠
arXiv:2605.16023v1 Announce Type: cross Abstract: LLM-as-a-judge has become the dominant paradigm for grading model outputs at scale, yet the same mod…