首个针对AI编程代理的恶意issue请求基准测试，揭示AI agent安全漏洞与鲁棒性挑战。
arXiv:2607.20759v1 Announce Type: cross Abstract: AI coding agents powered by LLMs are increasingly integrated into real-world software development, w…
开源SoC任务AI基准，直击芯片验证中36%误报率痛点，值得一试
Writeup: https://www.deeptempo.ai/blogs/the-36-percent-false-positive... Comments URL: https://news.ycombinator.com/item?id=48819132 Points: 2 # Comme…
Agent运行轨迹转化为训练数据，IFEVAL评测提升8.7个百分点，方差降低提升生产可靠性。
Trace-to-Training: how agent runs become learning data Every agent run is a data point. Most frameworks throw it away. WasmAgent keeps it — evaluated …
首个面向多智能体AI系统的提示注入基准测试工具，填补安全评估空白
Article URL: https://freyzo.github.io/deep-xpia/ Comments URL: https://news.ycombinator.com/item?id=48549498 Points: 1 # Comments: 0
新论文提出Agent Planning Benchmark，用于诊断和评估LLM Agent的规划能力，填补了相关评测空白。
arXiv:2606.04874v1 Announce Type: new Abstract: Planning is central to LLM agents: before acting, an agent must decompose goals, select tools, reason …
硅谷顶级风投Benchmark首次设立增长基金，加入20亿美元资本大军，投资策略迎来重大转向。
The legendary abandons its more than 20 year tradition of keeping its funds to about $425 million.
首个标准化EEG基础模型评估基准，覆盖多任务与多数据集，推动脑电AI研究规范化。
arXiv:2606.00815v1 Announce Type: new Abstract: Electroencephalography (EEG) supports a variety of brain-computer interface (BCI) tasks ranging from b…
一篇探讨理解边界处基准测试的学术论文，揭示AI评估新视角。
arXiv:2602.14307v3 Announce Type: replace Abstract: As frontier Large Language Models (LLMs) increasingly saturate new benchmarks shortly after they a…
多轮对话中记忆架构如何影响Text-to-SQL性能？基准与实证研究揭示关键设计差异。
arXiv:2605.26394v1 Announce Type: new Abstract: Multi-turn Text-to-SQL is central to enterprise analytics yet remains predominantly evaluated in singl…
首个大规模研究生级数学自动形式化基准，52k样本填补研究级数学空白。
arXiv:2605.14061v1 Announce Type: new Abstract: Current autoformalization benchmarks are largely focused on olympiad or undergraduate mathematics, whi…
首个专为遥感领域复杂推理设计的视觉语言基准VLRS-Bench，从认知、决策、预测三维度构建2000个高难度问答对，揭示现有MLLM在遥感推理上的显著瓶颈，为多模态AI在遥感应用的发展提供关键方向。
arXiv:2602.07045v2 Announce Type: replace-cross Abstract: Recent advancements in Multimodal Large Language Models (MLLMs) have enabled complex reasoni…
一个发人深省的发现：任务成功率和承诺完整性在LLM Agent评估中常常背道而驰。NeuroState-Bench通过人类校准的侧查询探针，揭示了传统结果导向评估的盲区——你信赖的“最佳模型”可能无法始终如一地兑现承诺。
arXiv:2605.01847v3 Announce Type: replace Abstract: Outcome-only evaluation under-specifies whether an evaluated agent profile preserves the commitmen…
首次用粒子物理实验复现任务来评估AI代理，测试长程推理与科学工具使用能力，填补现有基准缺乏真实科学复杂性的空白。
arXiv:2605.13950v1 Announce Type: cross Abstract: Autonomous language-model agents are increasingly evaluated on long-horizon tool-use tasks, but exis…