How to build a multi-agent system

Most teams reach for a multi-agent architecture one project too early. It is the most expensive shape you can build, it loses to a single agent on tasks with tight step-to-step dependencies, and its failures are harder to see because no human watches the middle.

This post covers when the spend pays off, how to write an orchestrator that delegates without duplicating work, how the tool surface underneath sets the ceiling on everything above it, and what breaks once the system runs in production. Much of the evidence comes from Anthropic's writeup of the Research feature, which is the most detailed public account of a multi-agent system running at scale.

The architecture is a spend decision

Before the design question comes the arithmetic. Anthropic measured agents consuming roughly 4x the tokens of a chat interaction, and multi-agent systems roughly 15x. That is the price of admission, and it is worth paying only where a better answer is worth 15 times more than an adequate one.

The payoff can be large. A lead agent on Claude Opus 4 with Claude Sonnet 4 subagents outperformed single-agent Claude Opus 4 by 90.2% on Anthropic's internal research eval. The number underneath that one deserves more attention: token spend alone explained about 80% of the variance in eval scores. Much of the gain is bought rather than designed, and a fleet of subagents is a way to spend tokens in parallel instead of in sequence.

Multi-agent therefore pays off when the task can absorb parallel token spend, which happens when:

• The work splits into branches that do not need to read each other's findings until the merge
• The relevant information exceeds one context window
• Breadth across many sources matters more than depth in one thread

It loses when agents need shared context, when each step depends on the last, or when the task has one right path to find. Anthropic names coding as a poor fit for exactly that reason: fewer genuinely parallel subtasks than research has. Their assessment of the coordination limits is blunt, that LLM agents "are not yet great at coordinating and delegating" to each other in real time, and building as though that were solved is how teams end up with a distributed system that argues with itself.

Marketing research sits on the favorable side of that line more often than not. "Which competitors gained search visibility this quarter, and on what" splits cleanly: one subagent per competitor, one per keyword cluster, none of them needing the others' output until the merge.

The orchestrator's real job is writing task descriptions

Duplicated work is the first failure you will see, and it traces back to the lead agent every time. Anthropic describes a run where one subagent researched a 2021 chip crisis while two others independently investigated 2025 supply chains. None of them made a mistake. The orchestrator handed out a topic instead of an assignment, and three reasonable agents read the same phrase three ways.

A subagent task needs four things written down:

• Objective: the specific question this agent answers, phrased so two agents could not both claim it
• Output format: the shape the finding comes back in, so the merge step is not free-text parsing
• Source guidance: which tools to use and which kinds of sources count
• Boundaries: what this agent explicitly does not cover, named against its siblings

The boundary line is the one teams skip and the one that prevents overlap. "Research competitor pricing pages" leaves a gap; "research pricing pages for competitor-a.com and competitor-b.com only, ignoring their blog and docs, which subagent 3 covers" does not.

Anthropic's method for finding these problems is worth stealing: run the exact system prompt and tools in the Console and watch the agent work step by step. Failure modes that survive a code review show up in the first two minutes of watching a transcript.

Put the fleet-sizing rule in the prompt

Left to its own judgment, an orchestrator over-serves. Anthropic saw agents spawn 50 subagents for queries a single call would answer, and search endlessly for sources that did not exist. Both come from the same gap: no stated relationship between question complexity and effort.

Their scaling rule goes in the lead agent's prompt directly:

• A fact lookup gets one agent and 3 to 10 tool calls
• A direct comparison gets 2 to 4 subagents making 10 to 15 calls each
• Open-ended research gets 10 or more subagents with divided responsibilities

Numbers this specific look arbitrary written down, and that is the point. The model needs a concrete anchor to calibrate against, not a principle like "scale effort appropriately" that it will interpret generously every time.

Pair the rule with a stopping condition per subagent. Anthropic lists agents continuing after they already had sufficient results as a recurring failure, which is the same over-service problem one level down the tree.

Design the tool surface before the fleet

Every agent in the fleet inherits whatever quality the tools have, which is why Anthropic puts agent-tool interfaces on the same footing as human-computer interfaces. An agent pointed at the wrong tool for the question is finished before it starts, and a bad tool description "can send agents down completely wrong paths".

Their fix scales better than hand-editing docs. They built an agent that uses a tool, hits its failure modes, then rewrites the tool's description to prevent them. Testing one tool dozens of times produced a description that cut task completion time for later agents by 40%.

Multi-agent multiplies the stakes. When a single agent picks the wrong tool, you lose one round trip. When ten subagents inherit the same misleading description, you lose ten, in parallel, and the merge step receives ten confidently wrong findings with no signal that anything went sideways.

Result shape matters as much as tool choice

The part that only shows up in multi-agent systems is the merge. A lead agent holding findings from eight subagents has to resolve conflicts between them, and it can only do that if the findings carry something to rank on.

Two subagents report position 3 for the same keyword. One read a live search result 30 seconds old; the other read an index estimate from last week. In untyped JSON those are the same claim, and the orchestrator picks whichever arrived first or reads better. That is not a prompting problem, and no amount of orchestrator instruction fixes a payload with nothing to arbitrate on.

Every OpenRush tool returns the same envelope, versioned ofe/1.0, and every claim inside carries its own provenance:

{
  "type": "seo.keyword_ranking",
  "subject": { "domain": "competitor-a.com", "keyword": "mcp server hosting" },
  "value": { "position": 3, "url": "https://competitor-a.com/hosting" },
  "provenance": {
    "source_class": "live_serp",
    "method": "serp_fetch",
    "confidence": 0.94,
    "observed_at": "2026-07-24T14:02:11Z",
    "cache_hit": false
  }
}

Now the merge has a rule it can follow: a live observation outranks an index estimate, a fresher observation outranks an older one, and a conflict the orchestrator cannot resolve becomes a verification task rather than a coin flip. The subject field pins each fact to stable entity ids, so findings from inspect_domain and inspect_serp join without string matching across eight subagent reports.

Truncation compounds the same way. A tool returning 25 rows of 412 without saying so gives each subagent a sample it reads as a survey, and the merged report reads like a complete picture assembled from eight complete pictures. Every OpenRush envelope carries a coverage block stating rows returned against rows available, so a subagent reading returned: 25, total_available: 412 knows to call export_dataset or to label its finding as partial.

Start wide, then narrow

Agents default to long, specific queries that return almost nothing, then conclude the information does not exist. Anthropic's correction mirrors how an experienced researcher works: open with short broad queries, read what actually exists on the topic, then narrow toward specifics once you know what is there.

This belongs in the subagent prompt, not the orchestrator's. The subagent is the one holding the search tool, and the instinct to over-specify fires on its first call.

Parallelism is where the wall-clock win comes from

Two layers of parallel calling do the work. The lead agent spins up 3 to 5 subagents at once instead of running them in sequence, and each subagent calls 3 or more tools at once instead of waiting on each result. Anthropic reports this cutting research time by up to 90% on complex queries.

The second layer is the one teams forget. A subagent that fetches five competitors serially spends five round trips where one would do, and that cost repeats across every subagent in the fleet.

Start evaluating at 20 queries

The instinct is to wait until you have a proper eval set, and it is wrong for a system this early. Anthropic started with about 20 queries, because changes at the beginning produce effects large enough to see at that sample size. A prompt fix that moves a metric from 30% to 80% does not need 500 test cases to be visible.

For scoring, one LLM judge call with one prompt beat the alternatives on consistency with human graders. Their rubric covers five things:

• Factual accuracy: do the claims match the sources
• Citation accuracy: do the cited sources support the claims made from them
• Completeness: are all requested aspects covered
• Source quality: primary sources over secondary ones
• Tool efficiency: the right tools, in a reasonable number of calls

The judge outputs scores from 0.0 to 1.0 plus a pass-fail grade, which makes hundreds of runs gradeable without a human reading them.

Human testers still catch what the rubric never asks about. Anthropic's testers found the agents preferring SEO-optimized content farms over authoritative primary sources, a bias no completeness score penalizes because the content farm answers the question. That failure is worth sitting with if you are building agents for marketing work specifically: the corpus your agent searches is the one an entire industry optimizes to game, and source quality has to be an instruction, not an assumption.

What breaks in production

Four problems need engineering that a prototype never surfaces, and each one gets worse with the number of agents running at once.

State and resumability

Agents hold state across long runs, so an error at minute 40 does not cost you minute 40, it costs the whole run. Restarting from zero is expensive enough that Anthropic built resumption from the point of failure instead. They pair deterministic safeguards, retries and checkpoints, with the model's own adaptability: telling an agent that a tool is failing lets it route around the failure instead of halting.

Tracing non-deterministic runs

The same query produces different execution paths on different runs, which makes reproduction unreliable as a debugging method. Anthropic added full production tracing of decision patterns and interaction structure, without reading conversation contents, and that is what made root causes findable.

The synchronous bottleneck

Most orchestrators run subagents synchronously, waiting for a full batch to finish before proceeding. That keeps coordination understandable, and it also blocks the entire system on the slowest subagent, because while a batch is in flight the lead cannot steer and the subagents cannot talk to each other. Asynchronous execution unlocks more parallelism and costs you result coordination, state consistency, and error propagation. Anthropic still runs synchronously and names the tradeoff openly.

Deploying without killing live runs

A multi-agent system is a stateful web of prompts, tools, and execution logic running more or less continuously, so there is no moment when nothing is in flight. Rainbow deployments shift traffic from old to new versions gradually, which keeps a correct code change from breaking runs mid-execution.

A multi-agent marketing run, end to end

A quarterly visibility review, sized as open-ended research, with the fleet rule and the merge rule set in advance.

1. The lead calls describe_capabilities once and puts the result in its plan, so it knows which tools are live, what each costs in credits, and which need a connected data source the account may not have.
2. discover_competitors refreshes the competitor set, because last quarter's list is a stale assumption rather than a starting point.
3. The lead spawns one subagent per competitor, each with an explicit boundary naming its domain and excluding the others. Each runs inspect_domain, inspect_search_visibility, and compare_keyword_coverage against your domain, and returns findings in a fixed shape.
4. A verification subagent takes the 10 keywords where competitor findings conflict or where the drop is largest and runs inspect_serp on each, confirming against a live result rather than an index estimate.
5. The lead merges, ranking claims by source_class and observed_at, and flags any finding whose coverage block shows truncation.

The output is a ranked list of visibility changes, each with a cited cause and an observed_at timestamp, which a human approves before anything ships. Subagent 3 crashing costs subagent 3, and the rest of the fleet finishes.

In our benchmarks, agents running on OpenRush spend 25% fewer tokens than the same agents pointed at the raw upstream API, because the model spends its budget deciding rather than parsing. In a fleet of ten subagents, that saving is per subagent.

What is still hard

Three problems are open, for us as much as for anyone building here.

The orchestrator cannot steer a running batch. Once subagents launch, a lead agent that learns something in minute 2 which would redirect all eight of them has no way to say so, and waits for work it already knows is wasted. Asynchronous coordination is the answer in principle and nobody has made it clean in practice.

Evaluating the merge specifically is unsolved. The rubrics score the final answer, which conflates good subagent findings with good synthesis. A run where seven subagents were right and the lead weighted the eighth incorrectly grades the same as a run where the research was bad.

Cost-aware planning barely exists. Exposing per-tool credit cost was the first step. An orchestrator that budgets a fleet against real prices, paying for a live observation only where a cheap estimate would change the answer, is still mostly hypothetical.

Start with two subagents

Take a research task you currently run as one agent, split it into two branches that do not need to read each other, and write both task descriptions with all four parts. Run it against 20 queries and compare against the single-agent baseline you already have. If the merged answer is not better, the task had dependencies you did not see, which is a more useful finding than a fleet of ten that looks impressive and duplicates itself.

Connect an agent to get the tool surface running, read How to manage AI agents that run without you for the scope and session questions that apply to every agent in the fleet, or see MCP, AI tooling, and the marketing data layer for why the envelope looks the way it does. </content>