破解AI看病难题：全新基准MedReaMM如何考核多模态大模型的专业诊断整合能力
arXiv:2608.22323v1 Announce Type: new Abstract: The application of Large Language Models (LLMs) to diagnostic decision-making has garnered growing int…
打破问卷、选择与生成文本混为一谈的评测假设，为LLM价值测量提供分层契约验证。
arXiv:2608.23411v1 Announce Type: new Abstract: LLM value studies often merge questionnaire ratings, pairwise choices, and values inferred from genera…
药物发现遇上智能体，看LLM评估系统如何借人类对齐把关可靠性
arXiv:2608.21057v1 Announce Type: new Abstract: Agentic large language model (LLM) systems are reshaping scientific workflows in chemistry and drug di…
揭秘专用裁判与共享评判的取舍之道，为LLM评估定制最优策略，实验详实、图表清晰。
arXiv:2607.27984v2 Announce Type: replace Abstract: Agentic systems generate outputs faster than human review. We contrast two LLM evaluator specializ…
用LLM当裁判评估大模型在5G故障诊断中的表现，为垂直领域落地提供新思路。
arXiv:2608.21021v1 Announce Type: cross Abstract: Real-world fault analysis in 5G and emerging 6G networks demands domain expertise to analyze free-te…
变异测试一招揪出LLM评测漏洞，Muteval让评估数据更可靠，开源免费直接上手。
Article URL: https://github.com/AshwinUgale/muteval Comments URL: https://news.ycombinator.com/item?id=49388918 Points: 1 # Comments: 0
针对开放式问答的评估难题，提出问题级专属评估标准，让LLM评分更贴合上下文需求。
arXiv:2603.23522v2 Announce Type: replace Abstract: Evaluating large language models (LLMs) on open-ended questions is difficult because response qual…
LLM裁判也有“偏心眼”：自我标签与他者标签会引发双向评估偏差，AI评测可信度再遭挑战。
arXiv:2608.18091v1 Announce Type: cross Abstract: As LLM-as-a-judge systems become increasingly widespread, self-preference in LLMs -- the tendency to…
给LLM判卷装上“安全气囊”：不确定时检索或弃权，还带可证明风险保证。
arXiv:2608.17994v1 Announce Type: new Abstract: Using LLMs as judges has become standard practice for evaluating model outputs at scale. This is parti…
单次推理评估多细则易受干扰，新方法用自蒸馏提升LLM裁判准确率与效率
arXiv:2608.14684v1 Announce Type: cross Abstract: LLM judges increasingly evaluate responses against fine-grained rubric checklists. When a sample req…
系统刻画前沿大模型的有害行为分布，为安全评估提供更细致的量化视角。
arXiv:2608.14577v1 Announce Type: cross Abstract: Frontier large language models (LLMs) safety evaluation has largely treated harmful generation as an…
大模型评估常浪费采样算力，这项研究用贝叶斯最优停止动态决定何时收手，兼顾精度与成本，是做评测必读的思路。
arXiv:2608.14425v1 Announce Type: new Abstract: LLM evaluations often use fixed sampling budgets, testing every item the same number of times even aft…
为评估大模型在科研协作中的诚信表现提供全新诊断框架，帮助研究者理性看待AI“共作者”。
arXiv:2608.12345v1 Announce Type: new Abstract: Language models are increasingly deployed as co-scientists, yet their ability to uphold research integ…
首个评估大模型科学推理可靠性的基准，直击AI代理在无下游验证场景中的信任危机。
arXiv:2608.11415v1 Announce Type: cross Abstract: Large language models are being proposed as agents in scientific workflows, in domains where no down…
LLM当裁判不靠谱？这项研究揭示无人类标注时AI评分的致命盲区，做评估必看。
arXiv:2503.05061v3 Announce Type: replace Abstract: Reliable evaluation of large language models (LLMs) is critical as their deployment rapidly expand…
如何评估大模型在移动端处理零散个人信息的能力？这项研究给出了全新评测基准与洞见。
arXiv:2608.10692v1 Announce Type: new Abstract: Large language models (LLMs) are increasingly deployed as mobile assistants, where a key challenge is …
首个面向金融推理的可验证基准，用可执行计算树生成测试，直击LLM结构化数据推理短板。
arXiv:2608.11047v1 Announce Type: cross Abstract: While existing benchmarks have made substantial progress in evaluating LLMs across STEM domains, fin…
基准测试越跑越贵，固定参数校准让跨模型对比更公平高效
arXiv:2604.12843v3 Announce Type: replace Abstract: The rapid release of both language models and benchmarks makes it increasingly costly to evaluate …
临床决策任务中，提示工程并非万能，三大模型实测效果颠覆直觉，调提示词前先看数据。
arXiv:2512.22966v2 Announce Type: replace Abstract: Large Language Models (LLMs) have demonstrated promise in medical knowledge assessments, yet their…
别再只盯模型准确率了——在推理流水线里，裁判LLM的“决策规则”才是决定结果质量的关键，这项研究给出了严谨边界。
arXiv:2608.07813v1 Announce Type: new Abstract: An LLM judge deployed inside a reasoning pipeline does not merely measure quality, it decides which an…