We're excited by the reaction to our research on scaling long-running autonomous coding.
我们对长时程自治编码规模化的研究引发的反应感到兴奋。
This work started as internal research to push the limits of the current models. As part of the research, we created a new agent harness to orchestrate many thousands of agents and observe their behavior. By last month, our system was stable enough to run continuously for one week, making the vast majority of the commits to our research project (a web browser). This browser was not intended to be used externally and we expected the code to have imperfections.
这项工作始于为试探当前模型极限而做的内部研究。作为研究的一部分,我们造了一个新的 agent harness 来编排数千个 agent 并观察它们的行为。到上个月,系统已稳定到能连续运行一周,为我们的研究项目(一个网页浏览器)贡献了绝大多数 commit。这个浏览器不打算对外使用,我们也预期代码存在瑕疵。
However, even with quirks, the fact that thousands of agents could work together to produce work that was almost entirely runnable without human intervention felt like a milestone worth sharing. Since then, we've continued our research, and we wanted to go more in-depth on how the harness was built.
但即便带着种种怪癖,数千个 agent 能协作产出几乎完全可运行的工作、期间无需人工干预——这本身就像一个值得分享的里程碑。此后我们继续了研究,这篇想把 harness 的构建过程讲得更深。
We're also making part of this research available to try for some users.
我们也把这项研究的部分功能开放给一些用户试用。
Background · 背景
Our research project started as a personal side project of mine.
这个研究项目最初是我的一个个人副业。
A browser felt like an interesting benchmark. It was complex enough to reveal limitations with frontier models, and there are many different subsystems that needed to work together.
浏览器是个有趣的基准:复杂到足以暴露前沿模型的局限,又有许多需要协同工作的子系统。
My initial plan was to support rendering web pages without JavaScript support. I started by prompting Opus 4.5, asking it to write a detailed plan for building a browser engine. I would repeatedly nudge it to "keep going" to see how far it would go on the plan.
我最初的计划是先支持渲染不带 JavaScript 的网页。我从让 Opus 4.5 写一份构建浏览器引擎的详细计划开始,然后反复催它「继续」,看它顺着计划能走多远。
This failed quickly. The model lost track of what it was doing, frequently stopped to proclaim success despite being far from it, and got stuck on complex implementation details. But it showed signs of deep knowledge and intelligence. It could write good code in small pieces.
很快就失败了:模型忘记自己在干什么,屡屡停下来宣告成功(实际远未完成),卡在复杂实现细节上。但它展现出深厚的知识与智能——小块小块地写,代码质量是好的。
The core issue was that the browser was too overwhelming of a task and needed to be broken down into subtasks. Next, I had the agent plan out a dependency graph of major work that agents could take up in parallel. Agents were manually spawned for tasks and nudged when they stopped. This increased the throughput, but the results weren't much better. Agents couldn't communicate with each other or provide feedback on the project as a whole. The system needed to be more dynamic.
核心问题是浏览器这个任务对单体来说太重,需要拆成子任务。下一步,我让 agent 规划一张可并行认领的工作依赖图,为各任务手动 spawn agent、停了就催。吞吐上去了,但结果没有好多少:agent 之间不能通信,也无法对项目整体给出反馈。系统需要更动态。
Meanwhile, GPT-5.1 (and later GPT-5.2) began showing better results for their ability to follow instructions precisely. This seemed like a good fit for long-running agents, so we updated our harness to use OpenAI models based on these experiments.
与此同时,GPT-5.1(以及后来的 GPT-5.2)在精确遵循指令上开始表现更好——这看起来正适合长时程 agent,基于这些实验我们把 harness 换成了 OpenAI 模型。
At this point, the harness could build a simple version of the web browser without JavaScript, but building a complete browser engine with one agent would be prohibitively slow.
到这一步,harness 已经能构建出不带 JavaScript 的简单浏览器;但用单个 agent 构建完整浏览器引擎,慢到不可行。
This started our next round of research. Could we spend 10x more on compute to get 10x more meaningful throughput?
这开启了下一轮研究:多花 10 倍(10x)算力,能不能换来 10 倍(10x)的有效吞吐?
From single to multi-agent · 从单 Agent 到多 Agent
We started a new repository with a simple Rust-based harness.
我们开了一个新仓库,装一个简单的 Rust harness。
Rather than dealing with the complexity of distributed systems, we instead ran the harness on a single large Linux VM (Virtual Machine) with lots of resources. To control the harness, we would SSH into the VM and use a simple terminal interface.
我们没有去碰分布式系统的复杂性,而是把 harness 跑在一台资源充裕的大型 Linux 虚拟机(VM)上。控制方式是 SSH 进虚拟机,用一个简单的终端界面。
We spent more time up front on proper observability into the system. We logged all agent messages, system actions, and command outputs, with timestamps so we could analyze and replay sessions. This was not only helpful for us to manually review, but also for piping back into Cursor to sift through large amounts of data and quickly find patterns.
我们把更多时间花在了前期搭建像样的可观察性上:带时间戳记录全部 agent 消息、系统动作与命令输出,从而能分析与回放会话。这不仅方便人工审查,也能喂回 Cursor 去筛大量数据、快速找模式。
### Self-coordination · 自协调 Our first multi-agent idea was the most simple: have agents with equal roles use a shared state file to see what others are working on, decide what to work on, and update the file.
第一个多 agent 方案最简单:给等权的 agent 们一个共享状态文件——看别人在做什么、决定自己做什么、更新这个文件。
We would be the least prescriptive about what to do and instead let the agents figure out how to self-coordinate. This failed quickly.
我们对「该怎么做」几乎不做规定,放手让 agent 自己琢磨怎么自协调。很快就失败了。
The coordination file quickly created more problems. Agents held locks for too long, forgot to release them, tried to lock or unlock when it was illegal to, and in general didn't understand the significance of holding a lock on the coordination file. Locking is easy to get wrong and narrowly correct, and more prompting didn't help.
协调文件很快制造出更多问题:agent 把锁握太久、忘了释放、在不合法的时机加锁解锁,总而言之不理解「持有协调文件的锁」意味着什么。锁这种东西容易做错、容错窗口极窄,加更多提示词也没用。
Locking also caused too much contention. 20 agents would slow to the throughput of 1-3 with most time spent waiting on locks. We tried giving agents a tool to explicitly wait on another agent's work, but they rarely used it. We also tried a lockless optimistic concurrency control approach, which reduced overhead but didn't eliminate confusion.
锁还带来过多争用:20 个 agent 的吞吐拖到 1-3 个的量级,大部分时间在等锁。我们试过给 agent 一个「显式等待另一 agent 工作」的工具,它们基本不用;也试过无锁的乐观并发控制(optimistic concurrency control),开销降了,困惑没消除。
The lack of structure between agents meant no single agent took on big, complex tasks. They avoided contention and conflict, opting for smaller and safer changes versus taking responsibility for the project as a whole.
agent 之间缺乏结构,意味着没有谁会认领大而复杂的任务——大家回避争用与冲突,选小而安全的改动,而不是为项目整体负责。
Adding structure and roles · 加上结构与角色
Next, we separated roles to gives the agents ownership and accountability:
下一步,我们把角色分开,给 agent 所有权与责任:
A planner would first lay out the exact approach and deliverables to make progress toward the user's instructions. This would be handed to an executor, who became the sole lead agent responsible for ensuring the plan was achieved completely. The executor could spawn tasks for workers, which provided linear scaling and throughput. For continued movement and accountability, an independent judge ran after the executor finished to determine whether it completed and whether another iteration should run. This resolved many coordination issues. Having a single role dedicated to owning and overseeing execution allowed workers to focus narrowly on their task while the overall system still delivered.
planner 先铺出确切的做法与交付物,向用户的指令推进;然后交给 executor——它是唯一的主导 agent,负责确保计划完整达成,可以为 worker 派生任务(带来线性扩展与吞吐)。为了持续推进与问责,executor 结束后由一个独立的 judge 判定是否完成、是否再跑一轮迭代。这解决了大量协调问题:有一个专职角色拥有并监督执行,worker 就能只盯自己的窄任务,而系统整体照样交付。
### Observing and hill-climbing · 观察与爬山法 Landing on this design required close observation of the system.
落到这个设计靠的是对系统的贴身观察与爬山法式的小步迭代。
If there was a major problem, it would tend to occur repeatedly and across many agents and tool calls. For example, we noticed there was too much contention because many agents were running git restore at once. We used Cursor to analyze logs and compare them against our prompts to understand why behavior didn't match expectations.
一个大问题往往会反复出现、横跨许多 agent 与工具调用。比如我们发现大量 agent 同时跑 git restore 造成过度争用——我们用 Cursor 分析日志、对照提示词,搞清楚行为为什么和预期不符。
Ultimately, we found this system to be bottlenecked by the slowest worker. It was too rigid.
最终我们发现这个系统的瓶颈是最慢的那个 worker——太刚性了。
Doing all planning upfront also made it hard for the system to dynamically readjust as new issues were discovered. Some agents would end up going in counterproductive directions, unable to self-correct until the next iteration of the loop.
全部计划前置也使系统难以在新问题出现时动态再调整:有些 agent 会走进反效果的方向,直到下一轮迭代才能自我纠正。
Continuous executor · 连续执行者
The next version removed the independent planner.
下一版砍掉了独立的 planner。
The executor could now also plan how to deliver the goal in addition to spawning tasks. Since it was the sole agent, it didn't need to write a plan anywhere, stick to one static unchanging plan, or rigidly wait for all workers.
executor 现在自己也能规划怎么交付目标,同时照旧派生任务。因为它是唯一的主导者,计划不需要写到任何地方、不需要固守一张静态计划、也不需要死板地等所有 worker。
### Ensuring freshness · 保持新鲜 To ensure agents across all roles wouldn't drift over long periods of time, we introduced freshness mechanisms:
为保证各角色的 agent 在长时间里不漂移,我们引入了新鲜度机制:scratchpad.md 要频繁重写而非追加;单个 agent 到上下文上限时自动总结;系统提示词里加入自反思与对齐提醒;鼓励 agent 随时转向、随时质疑既有假设。
The system was now highly dynamic and flexible: it could proactively explore code, reconsider decisions, manage workers, interleave tasks, and continuously reflect the latest information. We found agents were reasonably good at following instructions to completion, so the judge was removed to keep the system simple.
系统现在高度动态灵活:主动探索代码、重新考虑决策、管理 worker、交错任务、持续反映最新信息。我们发现 agent 把指令执行到底的能力已经不错,于是把 judge 也撤了,保持系统简单。
### Pathological behaviors · 病态行为 Despite these improvements, the continuous executor started exhibiting pathological behaviors. It would sleep randomly, stop running agents, do work itself, refuse to plan and spawn more than a few narrowly focused tasks, not properly merge worker changes, and claim premature completion.
尽管有这些改进,连续 executor 开始出现病态行为:随机睡觉、停掉手下的 agent、自己下场干活、拒绝规划也不肯多派任务(只发几个窄任务)、不好好合并 worker 的改动、过早宣告完成。
We found it was being given too many roles and objectives simultaneously, including: plan, explore, research, spawn tasks, check on workers, review code, perform edits, merge outputs, and judge if the loop is done. In retrospect, it makes sense it was overwhelmed.
我们发现它同时背了太多角色与目标:规划、探索、研究、派任务、盯 worker、审代码、亲自改、合并产出、判定循环是否结束。事后看,它被压垮很合理。
The final system design · 最终系统设计
The final design incorporates all of our learnings. A root planner owns the entire scope of the user's instructions. It's responsible for understanding the current state and delivering specific, targeted tasks that would progress toward the goal. It does no coding itself. It's not aware of whether its tasks are being picked up or by whom. When a planner feels its scope can be subdivided, it spawns subplanners that fully own the delegated narrow slice, taking full ownership in a similar way but only for that slice. This is recursive. Workers pick up tasks and are solely responsible for driving them to completion. They're unaware of the larger system. They don't communicate with any other planners or workers. They work on their own copy of the repo, and when done, they write up a single handoff that the system submits to the planner that requested the task.
最终设计纳入了全部教训:root planner 拥有用户指令的全部范围,负责理解当前状态、下发具体而有针对性的、能推进目标的任务;它自己不写代码,也不知道自己的任务有没有被领走、被谁领走。当某个 planner 觉得自己的范围可再分,就 spawn subplanner 完全拥有被委托的那一小片——用同样的方式持有所有权,只是范围缩小。这是递归的。Worker 认领任务,独自负责把它推到完成:不知道更大的系统存在,不与任何其他 planner 或 worker 通信,在自己的 repo 副本上干活,做完写一份 handoff,由系统提交给发起任务的 planner。
Interestingly, this does represent how some software teams operate today.
有意思的是,这确实就是今天一些软件团队的运作方式。
Subplanners increase throughput by rapidly fanning out workers while ensuring the whole system remains fully owned and responsible by an agent. This also helped with large projects and tasks where a single planner would otherwise get overwhelmed and develop tunnel vision.
subplanner 通过快速铺开 worker 提高吞吐,同时保证整个系统始终被某个 agent 完整拥有与负责。这对大型项目尤其有用——单一 planner 否则会被压垮、发展出隧道视野。
The handoff contains not just what was done, but important notes, concerns, deviations, findings, thoughts, and feedback. The planner receives this as a follow-up message. This keeps the system in continuous motion: even if a planner is "done," it continues to receive updates, pulls in the latest repo, and can continue to plan and make subsequent decisions.
handoff 里不只是「做了什么」,还有重要的备注、疑虑、偏差、发现、想法与反馈。planner 把它当作后续消息收到。这让系统持续运转:哪怕某个 planner 已经「完成」,它仍会不断收到更新、拉取最新 repo、继续规划并做后续决策。
All agents have this mechanism, which allows the system to remain incredibly dynamic and self-converging, propagating information up the chain to owners with increasingly global views, without the overhead of global synchronization or cross-talk.
所有 agent 都有这个机制,系统因此极其动态、自我收敛:信息沿链条上行给视野逐级变全局的所有者,而不需要全局同步或互相串线的开销。
### Removing the integrator · 砍掉整合器 We originally added an integrator for central globally-aware quality control and to remove contention from too many workers trying to push, rebase, resolve conflicts, and merge simultaneously.
我们最初加了一个 integrator(整合器):做中央的、有全局视野的质检,同时把「太多 worker 同时 push、rebase、解冲突、merge」的争用收拢到一处。
It quickly became an obvious bottleneck. There were hundreds of workers and one gate (i.e. "red tape") that all work must pass through. We tried prompt changes, but ultimately decided it was unnecessary and could be removed to simplify the system.
它很快成了显而易见的瓶颈:数百个 worker,全部工作要过同一道闸门(官僚流程)。我们试过改提示词,最终认定它没有必要,砍掉以简化系统。
Throughput and tradeoffs · 吞吐与取舍
The system peaked at ~1,000 commits per hour across 10M tool calls over a period of one week. Once the system started, it didn't require any intervention from us.
系统在一周内跨越 1000 万(10M)次工具调用,峰值约 1,000 commits/小时。一旦启动,不再需要我们任何干预。
There were intentional tradeoffs to achieve this throughput.
为达到这个吞吐,有一些有意为之的取舍。
### Commit correctness · Commit 正确性 When we required 100% correctness before every single commit, it caused major serialization and slowdowns of effective throughput. Even a single small error, like an API change or typo, would cause the whole system to grind to a halt. Workers would go outside their scope and start fixing irrelevant things. Many agents would pile on and trample each other trying to fix the same issue.
当我们要求每个 commit 前都 100% 正确,就造成严重的串行化与有效吞吐骤降:哪怕一个小错(API 改名、拼写错)都会让整个系统停摆——worker 越出边界去修无关的东西,一堆 agent 涌向同一处互相踩踏。
This behavior wasn't helpful or necessary. Allowing some slack means agents can trust that other issues will be fixed by fellow agents soon, which is true since the system has effective ownership and delegation over the whole codebase. Errors arise then get fixed quickly. The error rate remains small and constant, perhaps rarely completely clean but steady and manageable, not exploding or deteriorating.
这种行为既无益也无必要。留一些松弛,意味着 agent 可以相信别的问题很快会有同伴修掉——这是真的,因为系统对整个代码库有有效的所有权与委托。错误出现,随即被修掉;错误率小而恒定,也许从不彻底干净,但稳定可控,不爆炸也不恶化。
This may indicate that the ideal efficient system accepts some error rate, but a final "green" branch is needed where an agent regularly takes snapshots and does a quick fixup pass before release.
这也许说明:理想的高效系统接受一定的错误率,但需要一条最终的 green branch——由一个 agent 定期打快照、在发布前做一轮快速修缮。
### Synchronization overhead · 同步开销 Sometimes multiple agents touch the same file or refactor the same code. Instead of trying to stamp these out completely or overengineer a solution, we accept some moments of turbulence and let the system naturally converge and settle over a short period of time.
有时多个 agent 会碰同一个文件、重构同一段代码。我们没有试图彻底消灭或过度工程化,而是接受一些湍流时刻,让系统在短时间内自然收敛、平息。
This spends some extra tokens and creates local contention, but keeps the system overall simpler: easier to align models and not overwhelm them, easier to manage and observe, less friction, and better global productivity. It also avoids overly complex approaches.
这多花一些 token、制造局部争用,但让系统整体更简单:更容易让模型对齐不超载、更容易管理与观察、摩擦更少、全局生产率更高——也避免了过度复杂的方案。
Infrastructure learnings · 基建经验
Each multi-agent run ran on its own large machine with ample system resources, to avoid premature complexity around distributed systems. This was a good fit, as most runs would peak at several hundred agents, which typically saturated but did not overprescribe these machines. This architecture made it easier to observe system metrics, and share and copy state where necessary.
每次多 agent 运行都跑在自己的一台大型机器上、系统资源充裕——避免过早引入分布式系统的复杂性。这很合适:多数运行峰值在数百个 agent,通常会吃满但不超出这些机器。这个架构也让观察系统指标、必要时共享与复制状态更容易。
After limiting RAM usage of agents, the disk became the hotspot. Especially with a monolith project, hundreds of agents compiling simultaneously would result in many GB/s reads and writes of build artifacts. This had a significant impact on the overall throughput of the harness, which was an interesting lesson: the project structure, architectural decisions, and developer experience can affect token and commit throughput, simply because working with the codebase (e.g. compilation) dominates time, instead of ideally thinking and coding.
限制了 agent 的内存用量之后,磁盘成了热点。单体项目尤其明显:数百个 agent 同时编译,构建产物的读写达到 GB/s 量级,严重拖累 harness 整体吞吐。这是个有意思的教训:项目结构、架构决策与开发者体验会影响 token 与 commit 吞吐——因为与代码库打交道(比如编译)占据了时间大头,理想中的「思考与写码」反而退居次位。
There were also constraints and inefficiencies in the general development environment: things that make sense or aren't significant for a single user workspace, can stick out with hundreds of agents doing the same thing on one machine. One trivial way to solve this is to give each agent its own machine. But there are interesting low-hanging opportunities for large efficiency gains just by rethinking and redesigning some of these primitives and tools.
通用开发环境里还有各种约束与低效:对单用户工作区合理或无足轻重的东西,在数百个 agent 于同一台机器上齐干时就会扎眼。最平凡的解法是给每个 agent 配一台机器;但只要重新思考、重新设计其中一些原语与工具,就有触手可及的大幅效率提升机会。
For example, many tools like Git and Cargo use shared locks, largely as a simple concurrency control mechanism. Could bringing well-established mechanisms from concurrent systems like databases make these work just as well in multi-agent systems? All agents have their own copy of the repo, but most files and artifacts are identical; could adding simple copy-on-write and deduplication features, found in more sophisticated production storage systems, bring similar easy wins to a typically "single-user" system without building separate infrastructure?
例如 Git 和 Cargo 这类工具大量使用共享锁——基本是当作简单的并发控制机制。把数据库等并发系统里成熟的机制搬过来,能不能让它们在多 agent 系统里同样好使?所有 agent 各有一份 repo 副本,但大多数文件与产物是相同的;给这种典型「单用户」系统加上写时复制(copy-on-write)与去重这些高级生产存储系统里的普通特性,不另建基础设施,是不是同样能白捡效率?
Specifying intent to agents · 向 Agent 表达意图
Instructions given to this multi-agent system were very important.
给这个多 agent 系统的指令非常重要。
Initially, we didn't make them our primary goal, but instead aimed for a stable and effective harness. But the significance of instructions became apparent quickly. We were essentially interacting with a typical coding agent, except with orders of magnitude more time and compute. This amplifies everything, including suboptimal and unclear instructions.
一开始我们没把指令当主攻目标,先求一个稳定有效的 harness。但指令的重要性很快显形:我们本质上还是在和一个普通的 coding agent 打交道,只是时间与算力大了数量级——一切都被放大,包括不优与不清的指令。
Spending more time on the initial instructions makes sense. Ultimately, agents are still agents: trained to follow your instructions strictly, go down those paths, not change or override them, even if they're bad.
在初始指令上多花时间是划算的。说到底,agent 还是 agent:被训练来严格遵循你的指令、沿着那些路径走,不去更改或推翻——哪怕指令本身是坏的。
We wanted to see success in our research projects, so we altered our initial instructions as the project and harness evolved. We were learning how to build a browser alongside learning how to operate this new multi-agent system, and could see poor or underspecified specifications reflected in the quality of the outputs, which was not due to the harness itself. The harness was merely following our instructions exactly.
我们希望研究项目成功,于是随项目与 harness 的演进不断修改初始指令。我们一边学怎么造浏览器、一边学怎么运营这套新的多 agent 系统,并亲眼看到贫弱或欠规范的规格如何反映在产出质量上——那不是 harness 的问题,harness 只是在精确执行我们的指令。
Some examples from the browser project: initially, the instructions focused on implementing specs and squashing bugs. Instructions like "spec implementation" were vague enough that agents would go deep into obscure, rarely used features rather than intelligently prioritizing. We assumed implicitly that there were performance expectations within user-friendly bounds. But it took explicit instructions and enforced timeouts to force agents to balance performance alongside other goals. For complex parts of the system, agents may write code that has memory leaks or causes deadlocks. Humans would notice this, but it wasn't always obvious to agents. Explicit process-based resource management tools were required to allow the system to gracefully recover and be more defensive.
浏览器项目里的几个例子:最初指令聚焦「实现 spec、消灭 bug」。「实现 spec」这种说法模糊到 agent 会一头扎进冷门罕用的特性深处,而不是聪明地排优先级。我们默认存在「用户友好范围内」的性能预期——但必须靠显式指令与强制超时,agent 才会把性能与其他目标平衡起来。系统的复杂部分,agent 可能写出内存泄漏或死锁的代码——人会注意到,agent 不总是看得出来;需要显式的进程级资源管理工具,让系统能优雅恢复、更有防御性。
Our first version of the simple browser without JavaScript converged on an architecture that was unfit to evolve into a full browser. This was a failure of the initial specification.
第一版不带 JavaScript 的简单浏览器收敛出了一个无法演进成完整浏览器的架构——这是初始规格的失败。
Similarly, while the agents were told the project was a browser from scratch, they still pulled some dependencies they could've implemented themselves, or used as temporary scaffolding while proper implementation was underway. This was oversight in instructions. A later run explicitly laid out the dependency philosophy and which libraries must not be used, which corrected this.
同样,虽然告诉了 agent 这是从零自造浏览器,它们仍会拉取一些本可自己实现的依赖,或把依赖当临时脚手架用——这是指令的疏漏。后来一次运行显式写明了依赖哲学与禁用库清单,问题纠正了。
That later run also did a major restructuring into many self-contained crates, moving away from a monolith. The repo was in a heavily broken state, but the multi-agent system converged toward working code in a few days. This showed the system has strong capability to work collaboratively and intelligently, holding across totally broken states instead of degrading further or getting stuck. This run also spent far less time waiting for compilation, running at multiple times more throughput than previously.
那次后来的运行还做了一场大重构:从单体转向许多自包含的 crate。仓库当时处于严重损坏状态,但多 agent 系统几天内就收敛回可工作的代码——说明系统有很强的协作与智能能力,在彻底损坏的状态下依然撑得住,而不是继续恶化或卡死。这次运行的编译等待时间也大幅下降,吞吐是之前的数倍。
Architecture and instructions matter. Agents have immense engineering skill but will follow instructions to the end, good or bad. Finding the balance between overly narrow metrics and unstructured freedom was tricky, as was knowing what was obvious versus what needed explicit mention.
架构与指令都重要。agent 有强大的工程能力,但会把指令跟到底——无论好坏。在过窄的指标与无结构的自由之间找平衡很微妙,判断「哪些不言自明、哪些必须显式写明」同样微妙。
All of this indicates the importance of eliciting, specifying, and understanding intent, which becomes even more significant at this scale. Steerability and observability will be interesting research areas to continue exploring.
这一切都指向引导、表达与理解意图的重要性——在这个规模上愈发关键。可控性与可观察性会是值得继续探索的研究方向。
Optimizing prompts · 提示词优化
Prompting was a significant part of the evolution process.
提示词是整个演进过程的重要部分。
We found it was better to not instruct for things the model knows how to do, only things it doesn't know (e.g. multi-agent collaboration) or that are specific to the relevant domain (e.g. how to run tests, your deploy pipeline). Treat the model like a brilliant new hire who knows engineering but not your specific codebase and processes.
我们发现更好的做法是:模型本来会的就不要教,只教它不知道的(比如多 agent 协作)或领域特有的(比如测试怎么跑、部署管线长什么样)。把模型当成一位聪明的新员工——懂工程,但不懂你这套代码库与流程。
Constraints are more effective than instructions. "No TODOs, no partial implementations" works better than "remember to finish implementations." Models generally do good things by default. Constraints are defining their boundaries.
约束比指令更有效:「不许留 TODO、不许半成品」比「记得把实现做完」管用。模型默认就会做好事——约束是在给它们画边界。
Avoid checkbox mentality for higher-level or deeper tasks. Give detailed instructions about your intent, but remember giving specific things to do tends to make the model focus on achieving those rather than the wider scope. You also implicitly deprioritize unlisted things. Typically, it's better to let the model use its judgment and agency.
对更高层或更深的任务,避免打勾心态:可以把意图讲详细,但记住——给出具体待办清单会让模型聚焦于打完这些勾,而不是更宽的目标;没列的事还被你隐性降了优先级。通常,让模型用自己的判断与主动性更好。
We did find it useful to give concrete numbers and ranges when discussing quantity of scope. Instructions like "generate many tasks" tend to produce a small amount: conservative default, playing it safe, technically still following instructions. "Generate 20-100 tasks" conveys the intent is larger scope, it should be ambitious, and we observed very different wider behavior.
谈数量与范围时,给具体数字与区间确实有用。「生成很多任务」这类指令往往只产出少量任务——保守默认、求稳、技术上也算遵循了指令。「生成 20-100 个任务」传达的意图是更大的范围、应该有野心,我们观察到的行为立刻宽得多。
System design learnings · 系统设计心得
We established some principles from our research. The system should be anti-fragile. As we scale the number of agents running simultaneously, we also increase the probability of failure. Our system needs to withstand individual agents failing, allowing others to recover or try alternative approaches. Empirical over assumption-driven. We wanted to use data and observation to make adjustments, rather than coming in with assumptions about how it should work based on human organizations or existing system designs. Design for throughput explicitly. This meant trading off other aspects of coding, like accepting a small but stable rate of errors that requires a final reconciliation pass, instead of perfectly working code 100% of the time that would dramatically slow down the system.
从研究中我们确立了几条原则。系统应当反脆弱(anti-fragile):同时运行的 agent 越多,失败概率越高——系统要能承受单个 agent 失败,让其他 agent 恢复或改试别的路径。经验优先于假设:用数据与观察做调整,而不是带着「人类组织或既有系统设计应该如何」的假设进场。显式为吞吐设计:这意味着在其他方面做交换——接受小而稳定、需要最终一轮对账的错误率,而不是 100% 完美但会让系统慢到夸张的代码。
These systems tend to be elegantly simple when done right, but it wasn't clear which simple approach would work until we explored many different approaches. The current system design has been running with minimal overhead and provides linear scaling of token throughput in a useful manner. No major further iterations have been necessary on the harness.
做对的这类系统往往优雅而简单——但哪一种简单可行,是探索了很多不同方案之后才清楚的。当前设计以极小开销运行,token 吞吐以有用的方式线性扩展。harness 本身已无需再大改。
Conclusion · 结语
While taste, judgement, and direction came from humans, AI was a significant force-multiplier for rapidly iterating and exploring this research.
品味、判断与方向来自人类,而 AI 是快速迭代与探索这项研究的巨大力量放大器。
This has some resemblance to the "virtuous" AI loop, where AI is used to develop AI, and as models and agents and harnesses get better, it feeds into itself and accelerates faster and faster. We shape the tools which shape us.
这与「良性」AI 循环有几分相像:用 AI 开发 AI,模型、agent 与 harness 越变越好,又反过来滋养自身、加速再加速。我们塑造工具,工具再塑造我们。
There's a poetic resemblance in this research to how some software teams operate today. These models were not explicitly trained in this way, which suggests it's emergent behavior and possibly the correct way of structuring software projects after all.
这项研究与今天一些软件团队的运作方式有种诗意的相似。这些模型并没有被专门训练成这样——这暗示它是涌现行为,也可能本来就是软件项目正确的组织方式。
We will be continuing to research extremely long-running agents, with our findings informing the future of our product.
我们会继续研究超长时程 agent,这些发现将滋养我们产品的未来。