Language models can already help researchers search the literature, synthesize evidence, and work through complex questions. But scientific work places particular demands on these models—answers need to stay grounded in evidence, the models need to preserve what the evidence actually supports rather than quietly broadening a study's conclusions, and researchers need to be able to verify the final outputs.
语言模型已经能帮研究者检索文献、综合证据、推演复杂问题。但科学工作对这些模型有特殊要求：答案要钉在证据上；模型要守住证据实际支持的结论，而不是悄悄扩大研究的适用范围；研究者要能验证最终产出。
We see that in how scientists use Asta, our agentic platform for scientific work. Instead of simple keyword searches, users often bring substantial context and many constraints—for example, asking Asta to compare approaches across a body of literature while accounting for a particular method, population, or setting. Many also return to generated reports later, treating them as working research artifacts rather than one-off answers.
这一点体现在科学家怎么用 Asta——我们的科研 Agent 平台。用户很少做简单关键词搜索，而是带着大量上下文与多重约束来（比如让 Asta 在整片文献里比较方法，同时限定特定方法、人群或场景）。很多人还会回头再打开生成的报告——把它们当作可迭代的研究工件，而非一次性答案。
We wanted to help scientists generate cited reports faster, with a model they could download and run themselves. To do that, we tested whether a small, open model trained specifically for scientific report generation could match the report quality of the proprietary models we were using, while reducing generation time and serving costs.
我们想让科学家更快地生成带引用的报告，而且用一个他们能下载、自己跑的模型。为此我们检验：一个专为科学报告生成而训的小型开放模型，能否追上我们在用的专有模型的质量，同时砍掉生成时间与服务成本。
We built AstaBrief 8B, a model that turns a research question and retrieved literature excerpts into a cited report. AstaBrief is available in Asta's Generate a report feature today as Fast mode alongside Claude-powered Thinking mode, and we're also open-sourcing it and the training data so others can study, reproduce, and build on our approach.
我们造了 AstaBrief 8B——把研究问题加检索到的文献节选变成带引用报告的模型。今天它已上架 Asta 的「生成报告」功能，作为 Fast 模式与 Claude 驱动的 Thinking 模式并列；同时我们把它连同训练数据一起开源，供他人研究、复现和延伸。
Developing AstaBrief required tens of thousands of real research queries, citation-focused filtering, preference data, and a redesigned report-generation pipeline that writes the full report in one pass rather than section by section. The result is nearly an order-of-magnitude reduction in report generation time compared to the proprietary models we tracked—across the full Asta pipeline, Fast mode averages 51.1 seconds per report compared with 178.5 seconds for Thinking mode, about 3.5× faster.
研发 AstaBrief 动用了数万条真实科研查询、面向引用的过滤、偏好数据，以及重新设计的报告生成管线——一次直出全文报告而非逐节写。结果：与我们追踪的专有模型相比，报告生成时间近乎降一个量级——Asta 全管线里 Fast 模式平均每份报告 51.1 秒，Thinking 模式 178.5 秒，约快 3.5 倍。
Together, those efficiency gains made AstaBrief a useful test case for a broader goal: building open language models that can be adapted to the specific demands of scientific work.
这些效率增益让 AstaBrief 成为一个更宏大目标的有用试验案例：造能适配科研工作特殊需求的开放语言模型。
Open weights will also let institutions run AstaBrief on their own infrastructure, which is necessary when research questions reveal sensitive or unpublished work. Alongside the model weights, we're releasing an example workflow that researchers can adapt to create reports from their own PDFs, providing a starting point for local report generation.
开放权重还让机构能把 AstaBrief 跑在自己的基础设施上——当研究问题会暴露敏感或未发表工作时这是刚需。随模型权重，我们还发布了一个示例工作流，研究者可改造它从自己的 PDF 生成报告，作为本地报告生成的起点。
This post covers how we trained AstaBrief, what we learned about grounding it in scientific evidence, and which parts of our approach we think can carry forward to future models for science. Most of the training and evaluation described was completed in 2025, so the proprietary models used to generate training data and as comparison points reflect the frontier at the time. We haven't rerun the full evaluation against today's frontier models; the results below are best read as evidence about the particular training and system design choices we tested.
本文讲我们怎么训 AstaBrief、在「钉在科学证据上」这件事学到什么、哪些做法值得带进未来科研模型。所述训练与评测大多完成于 2025 年，用于生成训练数据和充当对比点的专有模型反映的是当时的前沿；我们没有对今天的前沿模型重跑完整评测——下面的结果最好当作「关于我们测过的那些训练与系统设计选择」的证据来读。
Training the model · 训练模型
Our goal with AstaBrief was to build an open-weights model with all the qualities that matter most for long-form scientific synthesis: answer quality, relevance, structure, and citation grounding. We started from Qwen3-8B and focused most of our effort on the post-training data, evaluation, and surrounding report-generation scaffolding.
AstaBrief 的目标是造一个开放权重模型，具备长篇科学综合最要紧的全部素质：答案质量、相关性、结构、引用落地。我们从 Qwen3-8B 起步，把大部分力气花在后训练数据、评测和配套的报告生成脚手架上。
Adapting general-purpose models for scientific work – and training new scientific models from scratch – is something we're exploring broadly across Ai2. Through NSF OMAI, a U.S. national initiative led by Ai2 to build fully open AI infrastructure and models for scientific discovery, our researchers are working directly with scientific communities to understand what they need from future open models and where today's general-purpose models fall short. That includes studying how needs differ across scientific fields and workflows, with more findings from that research to share in the future.
把通用模型适配科研（以及从头训科研模型）是 Ai2 全所范围的探索。通过 NSF OMAI（Ai2 牵头的美国国家计划，为科学发现造完全开放的 AI 基建与模型），我们的研究者直接与科学社区协作，弄清他们对未来开放模型的需要、以及今天的通用模型差在哪——包括研究需求如何随学科与工作流不同而不同，后续会有更多发现分享。
Recent work, including our DR Tulu, has shown that reinforcement-learning-based (RL) methods can improve long-form report generation for open-weights models, especially when judge models are involved in the training loop. We considered that path for AstaBrief, but ultimately focused on a simpler recipe built around supervised fine-tuning (SFT) and direct preference optimization (DPO).
近期工作（包括我们的 DR Tulu）表明基于强化学习（RL）的方法能改进开放权重模型的长篇报告生成，尤其当裁判模型进训练环时。我们考虑过这条路，但最终聚焦在监督微调（SFT）+ 直接偏好优化（DPO）的更简配方上。
RL-based training can be unstable and expensive. We wanted to see how far we could push report generation quality with a cheaper, more operationally manageable setup—one that's also easier to debug and iterate on.
RL 训练可能不稳且贵。我们想看用更便宜、运维更省心的方案能把报告生成质量推多远——这种方案也更好调试与迭代。
That made the quality of the training data especially important. Rather than relying on a more complex optimization method to compensate for noisy examples, we spent much of the project figuring out how to generate, select, and filter examples that actually demonstrated the report-writing behavior we wanted.
这让训练数据的质量变得格外重要：与其靠更复杂的优化方法去补偿噪声样本，我们花掉项目大部分时间琢磨怎么生成、挑选、过滤出真正示范了我们想要的那種报告写作行为的样本。
We also wanted AstaBrief to be faster so that users could get preliminary reports quickly that they could then iterate over in subsequent turns. For speed improvements, we decided to train AstaBrief to directly generate the final report in one pass given a user query and relevant retrieved snippets, bypassing the expensive snippet summarization and clustering stages our Claude-based Thinking mode uses and not writing out the answer section-by-section. Interestingly, we found it was possible to do so without sacrificing performance.
我们还想让 AstaBrief 更快，让用户先拿到初稿再逐轮迭代。提速的办法：训练它给定查询+检索段落后**一次直出最终报告**——跳过 Claude 版 Thinking 模式里昂贵的「摘要+聚类」阶段，也不逐节写。有意思的是，这样做并不牺牲表现。
Collecting SFT training data · 收集 SFT 训练数据
The training pipeline began with real user queries submitted through the system described in our paper "Synthesizing scientific literature with retrieval-augmented LMs" and ScholarQA, the framework that now underpins Asta's Generate a report feature. Rather than training only on synthetic prompts or benchmark-style tasks, we wanted AstaBrief to learn from real queries from real scientists.
训练管线从真实用户查询起步（经我们论文《Synthesizing scientific literature with retrieval-augmented LMs》所述系统与 ScholarQA——现支撑 Asta 报告生成的框架）。我们不想只拿合成 prompt 或基准式任务训练，要让 AstaBrief 向真实科学家的真实查询学。
Our research suggests that scientists often ask different things of language models than users do of general-purpose chatbots or traditional search tools. In our analysis of hundreds of thousands of Asta queries, expert researchers frequently supplied substantial context, multiple constraints, and relationships between concepts rather than relying on short, keyword-style prompts.
我们的研究显示，科学家对语言模型的用法常与通用聊天/传统搜索不同：对数十万条 Asta 查询的分析里，专家研究者频繁提供大量上下文、多重约束和概念间关系，而不是依赖短促的关键词式 prompt。
More recent Asta user studies have also surfaced differences in how researchers want AI involved in their work—some are comfortable using models for ideation or experimentation, while others prefer a narrower role in synthesis, literature surveillance, or pattern-finding. Across those differences, participants want clearer source traceability, more visibility into what a model is doing, and greater control over the context it uses.
更近的 Asta 用户研究还浮出研究者对 AI 介入方式的分歧——有人放心让模型做构思或实验，有人只想要更窄的角色（综合、文献监控、找模式）。分歧之外有共识：参与者都想要更清晰的来源可溯、更多「模型在干什么」的可见性、以及对所用上下文更大的控制权。
We filtered the user logs we collected for quality, relevance, and privacy, stripping out beta-tester and bot traffic, dropping queries that were too short to be meaningful, and using an LLM-based filtering pass to catch non-English queries, non-scientific requests, and prompts containing personal information. That left a pool of 90K research-focused queries.
我们按质量、相关性与隐私过滤收集到的用户日志：剥掉内测与机器人流量、丢掉短到无意义的查询、再用一遍 LLM 过滤拦非英文、非科研和含个人信息的 prompt——留下 90K（9 万）条聚焦科研的查询。
For SFT, we generated full-report target outputs from the filtered queries using the multi-step ScholarQA pipeline behind Asta's report generation. The pipeline retrieved relevant literature, organized the material into sections, and used a backing report-generating model to synthesize the evidence into a cited report. We drew on a mix of proprietary systems: Claude 3.5 Sonnet, Claude 3.7 Sonnet, o3, o4-mini, and GPT-4.1. After quality filtering, this yielded 47K usable training examples.
SFT 侧：用 Asta 报告生成背后的多步 ScholarQA 管线，从过滤后的查询生成全文报告目标输出——管线检索相关文献、按节组织材料、再由后台报告生成模型把证据综合成带引用的报告。专有系统混用：Claude 3.5 Sonnet、Claude 3.7 Sonnet、o3、o4-mini 与 GPT-4.1。质量过滤后得 47K 可用训练例。
Creating DPO pairs · 构造 DPO 偏好对
DPO required a different kind of training data. Instead of a single target report per query, we needed pairs of reports with one preferred over the other. We built those pairs from a separate subset of queries not used during SFT data generation. One report per query came from the existing ScholarQA pipeline, typically backed by Claude 3.5 Sonnet or 3.7 Sonnet. The competing report was generated by feeding ScholarQA's retrieved literature excerpts to a different model: o3, o4-mini, DeepSeek-V3, or DeepSeek-R1, depending on the example. Two judge models – GPT-4.1 and DeepSeek-R1 – compared each pair and picked a winner. We ensured that LLM judges were aligned with human preferences (95% agreement) and only kept pairs where both judges agreed, which gave us a cleaner preference set and cut much of the noise that typically shows up in preference data generated at scale. After quality filtering, the final DPO dataset came to about 6K examples. Using multiple generators and requiring agreement between two judges gave us a relatively simple way to construct preference data without treating any single model's output or judgment as ground truth.
DPO 需要另一種训练数据：不是每查询一份目标报告，而是「一份优于另一份」的报告对。对子来自未用于 SFT 的独立查询子集——每查询一份出自现有 ScholarQA 管线（多为 Claude 3.5/3.7 Sonnet 生成），竞争报告由 ScholarQA 检索到的文献节选喂给不同模型生成（o3、o4-mini、DeepSeek-V3 或 DeepSeek-R1，视例而定）。两位裁判模型（GPT-4.1 与 DeepSeek-R1）比对每对并选出胜者。我们验证过裁判与人类偏好对齐（95% 一致），且**只保留双裁判一致的样本**——偏好集更干净，砍掉了规模化偏好数据里典型的大半噪声。质量过滤后 DPO 数据集约 6K 例。多生成器 + 双裁判一致，给了我们一个相当简单的偏好数据构造法——不把任何单一模型的输出或判断当真理。
Filtering data for better attribution · 为更好的归因过滤数据
Our main evaluation target was SQABench-CS2, a set of 200 user-written computer science research questions. We tracked four metrics throughout the development of AstaBrief: Rubric score, which measures how much necessary content is covered by the report. Answer precision, which measures whether each paragraph is relevant to the question. Citation precision, which measures whether each citation supports the claim it's attached to. Citation recall, which measures whether the report's claims are fully supported by the citations provided.
主评测目标 SQABench-CS2——200 道用户亲笔的计算机科学研究问题。全程追踪四指标：**评分表分**（必要内容覆盖多少）、**答案精度**（每段是否切题）、**引用精度**（每条引用是否支撑它所附的论断）、**引用召回**（报告论断是否被所引材料完全支撑）。
For our final model, we also ran secondary evaluations: DeepScholarBench, a 63-query benchmark for long-form research synthesis built from recent ArXiv papers, and two separate pairwise evaluations against reports generated by the Claude-powered pipeline—an LLM-judged comparison on SQABench-CS2 and a small human study.
终版模型另跑了两轮副评：DeepScholarBench（63 题、基于近期 ArXiv 论文的长篇研究综合基准），以及两场对 Claude 管线报告的成对对比——SQABench-CS2 上的 LLM 裁判对比和一场小型人类研究。
A report can sound polished and complete while meandering from the question or attaching citations to claims from which the underlying evidence doesn't follow. For scientific synthesis, we needed to measure those behaviors separately. But citation support is only part of scientific faithfulness—a model can cite the right study and still make a stronger claim than the study itself supports. This can happen in subtle ways, for example, turning a finding about a particular sample into a generic claim about an entire population, shifting a result reported in the past tense into a present-tense statement that sounds more universally true, or turning a descriptive finding into a recommendation for what clinicians, policymakers, or researchers should do.
一份报告可以听着光鲜完整，实际既跑题又给论断挂了证据不支持的引用——科学综合必须分开度量这些行为。但引用支撑只是科学忠实性的一半：模型可以引对论文、却做出比该研究本身更强的论断。手法可以很微妙——把特定样本的发现说成整个总体的通则；把过去时报道的结果改成现在时陈述（听着更普适）；把描述性发现变成给临床医生/政策制定者/研究者的行动建议。
Those kinds of generalizations are especially important for scientific report generation because each step can broaden the apparent scope of the evidence without introducing an obviously false statement. A cited sentence may therefore be technically related to its source while still overstating what researchers actually established. Our development metrics focused primarily on relevance, coverage, and citation grounding; a richer evaluation of scientific report writers should also test whether they preserve the scope and strength of the claims in their sources.
这类泛化对科学报告生成尤其要命——每一步都在扩大证据的表观适用范围，却不引入任何明显假话。一句带引用的话可以技术上与其来源相关、却仍夸大研究者实际确立的结论。我们的开发指标主要盯相关/覆盖/引用落地；更完善的科学报告评测还应检验模型是否守住了来源论断的范围与强度。
Our first SFT runs improved overall content quality, but they still lagged behind our Claude-powered report generation pipeline on answer precision and citation quality. In other words, the model got better at writing reports, but it still wasn't grounded in evidence as consistently as we needed for scientific synthesis.
首轮 SFT 提升了整体内容质量，但答案精度与引用质量仍落后 Claude 管线——模型更会写报告了，但还没稳定地钉在证据上，达不到科学综合的要求。
That pushed us to spend more time on data quality. We tested four statistics-based filters to identify weaker synthetic training examples: Output-to-input token ratio. Answers with very high ratios were often noisy because they were generating a lot of text from too little evidence. Citation relevance. For each synthetic report in the training set, we averaged the retrieval relevance scores of its cited papers. Low averages suggested the report was relying too heavily on lower-ranked evidence. Citation density. We measured the share of statements that had at least one citation. Low-density reports often had large stretches of unsupported text. Citation diversity: We measured the share of papers cited in the answer, given the set returned by the Claude-powered report retrieval pipeline. Low scores suggested the report was overly reliant on a few papers.
这逼我们回头抠数据质量。我们测了四个统计过滤信号来识别较弱的合成训练例：**输出/输入 token 比**——比值畸高的答案常是噪声（证据太少话太多）；**引用相关度**——对训练集每份合成报告取其引用论文的检索相关度均分，低分说明过度依赖排名靠后的证据；**引用密度**——至少带一条引用的陈述占比，低密度报告常有大段无支撑文本；**引用多样性**——答案引用的论文占 Claude 检索管线返回集合的比例，低分说明过度依赖少数论文。
The strongest gains came from filtering out synthetic reports with low citation density; more aggressive filtering, filter combinations, and learning-rate sweeps didn't add meaningful gains.
**最大的增益来自滤掉低引用密度的合成报告**；更激进的过滤、过滤器组合、学习率扫描都没带来有意义的增益。
That was one of the clearest lessons from the project: more elaborate filtering wasn't necessarily better. A relatively simple signal – whether the synthetic reports consistently cited their claims – was more useful than several more complicated combinations we tried. Scientific specialization, in other words, isn't necessarily a matter of adding more scientific text to pretraining; the composition and quality of post-training data and whether it demonstrates behaviors like grounding and attribution can materially change how the resulting model performs.
这是全项目最清晰的教训之一：更精巧的过滤不一定更好——「合成报告是否始终给论断配引用」这个相对简单的信号，比我们试过的几个更复杂组合都有用。换言之，科学专精未必靠预训练加更多科学文本；后训练数据的构成与质量、它是否示范了落地与归因这类行为，都能实质改变成品模型的表现。
That focus on grounded, useful output also lines up with what we've heard in Asta user research. Participants note that generating more text isn't necessarily more helpful; they want concise synthesis and enough source traceability to review and verify results without wading through unnecessary outputs.
这种「钉在证据上的有用输出」也与 Asta 用户研究的反馈一致：参与者指出文字多不等于有用——他们要精炼综合、要足够的来源可溯性，能审阅核验结果而不必蹚过冗余输出。
Once we had a stronger SFT checkpoint, we ran DPO training on top of it. That stage pushed performance further, bringing AstaBrief within range of the Claude-powered report pipeline in Asta and DR Tulu on report generation.
拿到更强的 SFT 检查点后，我们在其上跑 DPO——这一步把表现再推一截，让 AstaBrief 在报告生成上进入 Asta 的 Claude 管线与 DR Tulu 的区间。
Validating the approach · 验证这条路线
Because this model was intended to work as part of our agentic Asta report generation framework (not necessarily as a standalone model), our main question was whether AstaBrief could preserve the report qualities we cared about while enabling a substantially faster and cheaper report-generation pipeline. In other words, we weren't only asking whether the model could match a stronger proprietary model on individual benchmarks; we wanted to know how much of that quality we could retain with a much simpler system.
这个模型定位是 Asta Agent 报告生成框架的一部分（不必然是独立模型），所以核心问题是：AstaBrief 能否在大幅提速降本的管线里保住我们在乎的报告素质——不是问它能不能在单个基准上追平更强的专有模型，而是问用简单得多的系统还能留住多少质量。
In the evaluations we used during development, AstaBrief was competitive with the Claude-powered pipeline and DR Tulu across several measures of answer and citation quality. The chart below shows the LLM-judged comparison—in a separate 14-question human study, three scientific researchers each contributed 4-5 questions and ranked reports from the three systems on overall preference, completeness, relevance, organization, and citation accuracy (with ties allowed). On overall preference, DR-Tulu wins, but two of the three researchers prefer AstaBrief over other systems on citation accuracy metrics, demonstrating the utility of our SFT data quality filters.
开发期评测里，AstaBrief 在答案与引用质量的多个维度上与 Claude 管线和 DR Tulu 有来有回。下图为 LLM 裁判对比；另一场 14 题人类小研究里，三位科研人员各出 4-5 题，按总体偏好/完整性/相关性/组织/引用准确度给三个系统的报告排序（允许并列）。总体偏好上 DR-Tulu 胜，但**三位中两位在引用准确度上首选 AstaBrief**——SFT 数据质量过滤的效用实证。
These numbers are best read as validation of the engineering approach at the time we developed it, rather than as a claim about where this particular base model sits relative to today's frontier. The model ecosystem moves quickly—the data construction, attribution filtering, and serving lessons are the pieces we expect to generalize.
这些数字最好读作「研发当时那条工程路线的验证」，而非「这个基座在今天前沿里的位置」的宣言。模型生态迭代太快——数据构造、归因过滤与服务经验才是我们预期可泛化的部分。
Validating the usefulness of AstaBrief in Asta, Fast mode has shown encouraging early usage. Among 374 Asta users who've tried it, 29.1% have used it for two or more days, and users on average generate 3.67 report threads with it. Twenty-three percent of users who tried Fast mode continued using it and never switched back to Thinking mode for future threads. An additional 18% switched between Fast and Thinking modes depending on their goals, using Fast mode for ~40% of their threads. While feedback is generally too sparse to draw strong conclusions, we see that Fast mode receives positive feedback at a similar rate as Thinking mode (84.2% versus 85.2%).
在 Asta 内验证实用性：Fast 模式早期使用数据令人鼓舞——374 个试用用户里 29.1% 用了两天以上，人均生成 3.67 个报告线程；**23% 的试用用户后续再没回 Thinking 模式**，另有 18% 按目标在两模式间切换（约 40% 线程用 Fast）。反馈虽稀疏不足以强结论，但 Fast 模式的好评率与 Thinking 相当（84.2% 对 85.2%）。
Where this goes next · 下一步
Asta's report generation is the first production use of AstaBrief, giving researchers an open-weights Fast mode alongside the existing Thinking mode. Because the model is open weights, institutions can deploy it on their own hardware, including behind their own firewall, without relying on a proprietary model API for report generation. In Asta, that also means we can study and improve this part of the report generation pipeline directly while preserving Thinking mode as an option for more compute-intensive tasks.
Asta 报告生成是 AstaBrief 的首个生产用途——给研究者一个开放权重的 Fast 模式与既有 Thinking 并列。因为权重开放，机构可部署在自己的硬件上（包括自家防火墙后），不依赖专有模型 API。在 Asta 内部，这也意味着我们能直接研究并改进报告管线的这一环，同时保留 Thinking 处理更吃算力的任务。
There's more to do. We're exploring more fine-grained preference learning, stronger RAG-plus-RL approaches, multi-turn and multi-tool capabilities, additional scientific data sources, and query decomposition. We're also interested in evaluations that go beyond whether a claim has a supporting citation to ask whether a model preserves the evidentiary scope of its sources—both to better capture the quality of the report as a research artifact and to ask whether the model turns sample-specific findings into broad generalizations or descriptive results into recommendations. AstaBrief is one experiment in a longer line of work on language models for science, from ScholarQA and DR Tulu to future versions of Olmo beginning to take shape now. The lessons here – especially around training data, filtering, and evaluation – can help inform what we build next.
还有更多要做：更细粒度的偏好学习、更强的 RAG+RL、多轮多工具能力、更多科学数据源、查询分解。我们还关心超越「论断有没有支撑引用」的评测——问模型是否守住来源的证据范围（样本级发现有没有被说成总体通则、描述性结果有没有被变成建议），既为更好地度量报告作为研究工件的质量。AstaBrief 是「面向科学的语言模型」这条长线里的一次实验——从 ScholarQA、DR Tulu 到正在成形的未来版 Olmo。这里的教训——尤其训练数据、过滤与评测——将帮助我们决定下一步造什么。
Try Fast model today in Asta, or download AstaBrief from Hugging Face.
今天就在 Asta 里试 Fast 模式，或从 Hugging Face 下载 AstaBrief。