1
Separating signal from noise in coding evaluations
OpenAI揭秘热门编码基准测试的缺陷，重新审视AI模型评估的可靠性
A new analysis from OpenAI reveals issues in SWE-Bench Pro, a popular coding benchmark, raising concerns about reliability and accuracy in evaluating …
2
Show HN: Caliper – pass@k reliability testing for Claude Code and Codex skills
一个为Claude Code和Codex技能量身打造的pass@k可靠性测试开源工具，量化评估AI编码能力的真实水平。
Skills for Claude Code and Codex are hard to test. What I mean by hard is that there's no standard way to do it. You evaluate the skill once on someth…
3
HAI-Eval: Measuring Human-AI Synergy in Collaborative Coding
新研究提出人机协同编码评估系统，填补现有测试与LLM基准的空白，聚焦真实协作场景。
arXiv:2512.04111v2 Announce Type: replace-cross Abstract: LLM-powered coding agents are reshaping the development paradigm. However, existing evaluati…