1
Separating signal from noise in coding evaluations
OpenAI揭秘热门编码基准测试的缺陷，重新审视AI模型评估的可靠性
A new analysis from OpenAI reveals issues in SWE-Bench Pro, a popular coding benchmark, raising concerns about reliability and accuracy in evaluating …
2
SupraBench: A Benchmark for Supramolecular Chemistry
首个超分子化学AI基准测试，推动分子计算与新药研发
arXiv:2606.13477v1 Announce Type: cross Abstract: Supramolecular chemistry, which includes the study of non-covalent host-guest assemblies, has advanc…
3
What AI benchmarks miss about real-world performance
AI基准测试忽略的致命短板：买够GPU却卡在数据路径的突发瓶颈上。
Presented by F5 Enterprise AI teams have spent years solving for compute, securing GPU allocations, negotiating cloud capacity, and benchmarking train…
4
Can AI Reason Like an Urban Planner? Benchmarking Large Language Models Against Professional Judgment
大型语言模型能否像城市规划师一样推理？研究用专业判断基准测试AI，揭示AI与人类专家的能力边界。
arXiv:2606.11678v1 Announce Type: new Abstract: Problem, Research Strategy, and Findings: The rise of large language models (LLMs) raises a key questi…
5
ASyMOB: Algebraic Symbolic Mathematical Operations Benchmark
新基准ASyMOB用3.5万道符号数学题区分LLM的推理与模式记忆，直击大模型数学能力评估痛点。
arXiv:2505.23851v2 Announce Type: replace-cross Abstract: Large language models (LLMs) are increasingly applied to symbolic mathematics, yet existing …
6
These LLMs are the best at resisting Russian propaganda
最新基准测试显示，Anthropic的Claude模型在抵制俄罗斯宣传方面表现最优，Opus 4.7获得“杰出”评级。
Estonian government benchmark shows how dozens of models combat Russia's "strategic narratives."
7
Show HN: AI Model Benchmark for Crypto Price Predictions
实时追踪AI模型对加密货币价格预测的准确率，数据可视化对比各模型表现。
I've been running Crypto Predictions for over a month with a limited number of coins and models (some models in the same family the produce close resu…
8
PRAIB: Peer Review AI Benchmark of Behaviour of LLM-Assisted Reviewing
首个专门评估LLM辅助同行评审行为的基准，揭示AI评审的可靠性边界。
arXiv:2605.29815v1 Announce Type: new Abstract: The growing number of submitted papers has motivated the exploration of Large Language Models (LLMs) a…
9
LUMINA: A Multi-Vendor Mammography Benchmark with Energy Harmonization Protocol
首个多供应商乳腺X线摄影基准数据集LUMINA，引入能量协调协议，助力开发更鲁棒的AI诊断模型。
arXiv:2603.14644v3 Announce Type: replace-cross Abstract: Publicly available full-field digital mammography (FFDM) datasets remain limited in size, cl…
10
AI Cartography: Mapping the Latent Landscape of AI Benchmark Ecosystems
揭示AI排行榜得分背后的测量噪声，并提出框架量化潜在能力差异，颠覆你对基准测试的认知。
arXiv:2605.25272v1 Announce Type: new Abstract: While aggregate leaderboard scores drive AI development, they contain substantial measurement noise wh…
11
The Evaluation Trap: Benchmark Design as Theoretical Commitment
AI基准测试暗藏理论假设，窄化进步定义，警惕评估陷阱重塑能力概念
arXiv:2605.14167v1 Announce Type: new Abstract: Every AI benchmark operationalizes theoretical assumptions about the capability it claims to assess. W…
12
IPR-1: Interactive Physical Reasoner
看AI如何像人类一样通过观察和交互学习物理与因果，1000+异构游戏基准测试揭秘突破！
arXiv:2511.15407v3 Announce Type: replace Abstract: Humans learn by observing, interacting with environments, and internalizing physics and causality.…