The EvalEval Coalition is thrilled to share that the UK AI Security Institute (AISI) is using EvalEval's infrastructure to openly share evaluation results, supporting more reproducible and verifiable evaluation science.
EvalEval 联盟高兴地宣布：英国 AI 安全研究所（UK AISI）正在使用 EvalEval 的基础设施开放共享评测结果，推动更可复现、可验证的评测科学。
AISI and EvalEval have previously collaborated on research that began at a joint workshop alongside NeurIPS 2025, and feedback from the Institute has helped shape the Every Eval Ever (EEE) schema. This next phase of the collaboration puts that shared infrastructure into practice.
AISI 与 EvalEval 此前就有合作——始于 NeurIPS 2025 期间的一次联合工作坊，研究所的反馈也帮助塑造了 Every Eval Ever（EEE）schema。合作的下一阶段，就是把这套共享基建投入实用。
Why reproducible evaluation reporting matters · 为什么可复现的评测报告很重要
As AI deployment accelerates, evaluations are becoming increasingly important sources of evidence about model and system performance. Yet results are reported across many formats, platforms, and outlets, often without enough information to reproduce them. Running the evaluations again may itself be prohibitively expensive.
随着 AI 部署加速，评测正日益成为模型与系统性能的重要证据来源。然而结果散落在五花八门的格式、平台与出口上，常常缺少足以复现它们的信息——而把评测重新跑一遍，本身可能就贵到不可行。
EvalEval's mission is to improve this ecosystem through a shared reporting schema, Every Eval Ever, and an open platform, Evaluation Cards, that brings evaluation results and the information needed to interpret them into a common structure.
EvalEval 的使命是通过一套共享报告 schema（Every Eval Ever）与一个开放平台（Evaluation Cards）改善这个生态——把评测结果和解读它们所需的信息，装进统一的结构里。
This builds naturally on AISI's work to make evaluation more efficient through OptStop, more statistically rigorous through HiBayES, and more standardised in areas including transcript analysis and capability elicitation. Together, AISI and EvalEval are working to diagnose gaps in evaluation reporting and build shared infrastructure to close them.
这与 AISI 自身的工作自然衔接：用 OptStop 让评测更省算力、用 HiBayES 让统计更严谨、在转录分析与能力引导等方向推动标准化。双方正在一起诊断评测报告的缺口，并搭建共享基建来补上。
What AISI is sharing · AISI 开放了什么
Transcript-level transparency matters not only for reproducibility, but also for analysis and diagnosis. In this new phase of the collaboration, AISI is making publicly reported evaluation methods and findings available through Evaluation Cards where appropriate. The release includes verified results, context, and configuration information for the five benchmarks in the paper's main experiment:
转录（transcript）级的透明不只关乎可复现，也关乎分析与诊断。合作的新阶段里，AISI 把公开发布过的评测方法与发现，在合适处以 Evaluation Cards 形式开放。本次发布包含论文主实验五个基准的验证结果、上下文与配置信息：
HealthBench, FrontierMath, Humanity's Last Exam, SWE-Bench Pro, and Terminal-Bench 2.0.
HealthBench、FrontierMath、Humanity's Last Exam、SWE-Bench Pro、Terminal-Bench 2.0。
These results cover six frontier models: Claude Opus 4, Claude Opus 4.5, Claude Opus 4.6, GPT-5, GPT-5.2, and GPT-5.4. The release also includes results from two related cyber evaluations—Cyber CTFs and The Last Ones—which use a different, partially overlapping set of models. The data accompany AISI's paper, How Inference Compute Shapes Frontier LLM Evaluation, which studies how benchmark performance depends on inference-time compute and evaluation protocol.
结果覆盖六个前沿模型：Claude Opus 4、Claude Opus 4.5、Claude Opus 4.6、GPT-5、GPT-5.2、GPT-5.4。发布还包含两项相关网络安全评测——Cyber CTFs 与 The Last Ones——用的是另一组部分重叠的模型。数据随 AISI 的论文《How Inference Compute Shapes Frontier LLM Evaluation》一起发布，该论文研究基准表现如何依赖推理时算力与评测协议。
Performance on Humanity's Last Exam changes with evaluation protocol and inference compute. Each curve shows the cumulative share of attempted tasks solved within a given token count, using the earliest observed success per task. When models received correctness feedback from an oracle after each attempt, they continued to solve additional tasks as token use increased.
Humanity's Last Exam 上的表现随评测协议与推理算力而变。每条曲线展示在给定 token 数内解出的尝试任务累计占比（每任务取最早一次成功）。当模型每次尝试后收到 oracle 的正确性反馈时，它们随 token 消耗增加持续解出更多任务。
When results are openly released with setup information, researchers and practitioners can examine individual studies more closely and compare findings across the wider ecosystem. Where other reports lack these details, releases like AISI's provide verified reference points for interpreting evaluations in context—for example, by helping researchers understand how setup choices may influence reported performance. As more evaluators adopt EEE, open comparisons like these can support broader and more reliable meta-research.
结果连同设置信息一起开放时，研究者与从业者就能更细地审视单项研究、在整个生态里横向比较发现。当其他报告缺这些细节时，AISI 这样的开放发布提供了「带上下文解读评测」的验证参照点——比如帮研究者理解设置选择如何影响报告的性能。随着更多评测方采用 EEE，这类开放比较能支撑更广、更可靠的元研究。
AISI's Terminal-Bench 2.0 results alongside other reported evaluations for the same models, under different evaluation setups.
AISI 的 Terminal-Bench 2.0 结果，与同批模型在不同评测设置下的其他已发布评测并列展示。
We are excited about this adoption and look forward to further standardising and sharing evaluations with AISI and other AI evaluation organisations.
我们为这次采用感到兴奋，期待与 AISI 及其他 AI 评测组织进一步推进评测的标准化与共享。
Contribute to the shared mission · 参与共享使命
Model developers: report verified evaluation results. Evaluation developers: report benchmarks and run data using the Every Eval Ever schema. Evaluation, governance, and policy researchers: explore Evaluation Cards by benchmark or model, or use it to examine the state of evaluation reporting as a whole.
模型开发者：报告经验证的评测结果。评测开发者：用 Every Eval Ever schema 报告基准与运行数据。评测、治理与政策研究者：按基准或模型浏览 Evaluation Cards，或用它审视整个评测报告生态的现状。
About the EvalEval Coalition · 关于 EvalEval 联盟
The EvalEval Coalition is a research community developing scientifically grounded research and robust deployment infrastructure for the evaluation ecosystem. Its goal is to improve evaluation science, address the lack of consensus around documenting evaluation applicability and utility, and broaden coverage of the impacts that matter for scientific research and policy analysis.
EvalEval 联盟是一个研究社区，为评测生态开发有科学根基的研究与稳健的部署基建。目标是改进评测科学、解决「如何记录评测适用性与效用」缺乏共识的问题，并拓宽对科研与政策分析真正重要的影响的覆盖。
The coalition's flagship projects include Every Eval Ever, a shared schema and repository for evaluation results, and Evaluation Cards, which combines benchmark metadata, evaluation-run data, and model metadata into interpretable records. Together, they make it easier to understand when apparently similar scores were produced under meaningfully different conditions.
联盟的旗舰项目包括 Every Eval Ever（评测结果的共享 schema 与仓库）和 Evaluation Cards（把基准元数据、评测运行数据、模型元数据合成可解读的记录）。两者合力，让人更容易看清：看起来相近的分数，是在怎样实质不同的条件下产生的。
About the UK AI Security Institute · 关于英国 AI 安全研究所
The UK AI Security Institute is a research organisation within the UK government's Department for Science, Innovation and Technology. Its mission is to equip governments with a scientific understanding of the risks posed by advanced AI. AISI conducts research and builds infrastructure to understand advanced AI capabilities and impacts, develop and test mitigations, and inform policy.
英国 AI 安全研究所（UK AISI）是英国政府科学创新与技术部内的研究机构，使命是让政府对先进 AI 的风险具备科学理解。AISI 开展研究并建设基础设施，以理解先进 AI 的能力与影响、开发并测试缓解措施、支撑政策制定。
Further reading · 延伸阅读
How Inference Compute Shapes Frontier LLM Evaluation; HiBayES: Improving LLM evaluation with hierarchical Bayesian modelling (and paper); OptStop paper; Every Eval Ever; Evaluation Cards.
《How Inference Compute Shapes Frontier LLM Evaluation》；《HiBayES：用分层贝叶斯建模改进 LLM 评测》（及论文）；OptStop 论文；Every Eval Ever；Evaluation Cards。