Multi-Agent Orchestration Patterns: A Practical Guide

Splitting a task across multiple agents is usually justified by one of three things: the subtasks are genuinely independent and can run in parallel, different subtasks need different specialization or tooling, or a single agent's context window and attention cannot hold the whole problem at once. None of those justifications say anything about how the agents should be arranged — and the arrangement is what determines whether a failure in one agent stays contained, how much a run costs, how long it takes, and whether anyone can explain what happened afterward. Teams often skip this decision, let the topology emerge from whatever the first prototype happened to do, and only notice the cost once an incident forces them to reconstruct a multi-agent run they never designed to be reconstructable.

This guide treats topology as a first-class design decision with the same weight as model choice or retry policy. It compares three patterns — orchestrator/worker, peer-to-peer, and hierarchical supervisor trees — on the dimensions that actually matter in production: where a failure stops spreading, what the run costs and how predictably, how state moves between agents, what a retry means at each hop, and what you need to record to debug a bad outcome. It ends with a small, working orchestrator/worker implementation so the abstract trade-offs have a concrete shape to test against.

At a glance

This guide is for engineers building or reviewing a multi-agent system. It helps you choose between orchestrator/worker, peer-to-peer, and hierarchical topologies using failure isolation, cost, latency, and observability as the deciding evidence, so you can leave with a topology decision you can defend and a small working orchestrator/worker implementation to start from.

The coordination boundary framework

Use Shape → Bound → Route → Observe as the decision path for any multi-agent design:

1. Shape: match the topology to how the task actually decomposes — a plan with independent subtasks, a negotiation between equals, or a chain of delegated authority.
2. Bound: decide, before writing code, how far a single agent's failure is allowed to spread.
3. Route: choose how work, state, and context move across each agent-to-agent hop.
4. Observe: instrument every hop so the trace shape mirrors the control-flow shape, not the other way around.

Decision rule: If you cannot draw the failure boundary on a whiteboard before writing the first agent, the topology is not chosen yet — it is emerging by accident.

This article is deliberately narrower than "multi-agent systems" as a whole. It does not cover model selection, prompt design for individual agents, or long-running human-in-the-loop review queues in depth — debugging AI agents in production and AI agent reliability cover adjacent ground on those topics. The boundary here is coordination: how agents are arranged, how failure spreads between them, and what you need to record to explain a multi-agent run after the fact.

Three topologies, one decomposition problem

Every multi-agent system starts from the same question: this task is too large, too ambiguous, or too risky for one agent to do end to end, so how should the work be split? The three answers below cover most production systems. They are not mutually exclusive — a large system often nests one inside another — but each has a distinct shape, and that shape determines almost everything else in this article.

Orchestrator/worker

A single planner agent decomposes the task, dispatches bounded subtasks to specialist workers, and assembles their results. Workers do not talk to each other; every message goes through the orchestrator. This is the topology most teams reach for first, and for good reason: it maps directly onto a plan, and the orchestrator is a natural place to put budget checks, retries, and a final assembly step.

The orchestrator carries a real cost, though. It is a single point of coordination — if its planning logic is wrong, every worker downstream inherits that mistake, and if it goes down mid-task, in-flight work has no path forward without an explicit resume mechanism.

Peer-to-peer

Agents exchange messages directly with each other, with no central coordinator holding the full plan. This fits problems that are genuinely a negotiation or a conversation rather than a decomposition — for example, a drafting agent and a critic agent going back and forth until a document meets a bar, or two specialist agents reconciling conflicting data without deferring to a third party.

The appeal is flexibility: agents can loop, backtrack, and reach agreement without waiting for a coordinator to notice. The cost is that no single agent, and no single log, tells you the whole story. A peer-to-peer mesh trades a bounded failure for an emergent one — you gain adaptability and lose a place to stand when something goes wrong.

Hierarchical / supervisor tree

Nested supervisors each own a bounded branch of the work, delegating further down and reporting summarized results up. A root supervisor might own "handle this customer request," delegating to a billing supervisor and a technical supervisor, each of which delegates to narrower workers. This is orchestrator/worker applied recursively, and it is the natural choice once a single orchestrator's fan-out gets too wide to reason about, or once different branches need genuinely different policies (a billing branch with stricter approval rules than a FAQ branch, for instance).

The added structure is not free. Every layer of supervision adds a hop, a summarization step, and a place where a supervisor's imperfect summary can lose information the layer above needed.

When each pattern earns its complexity

Failure isolation and blast radius

The single most consequential difference between these topologies is how far one agent's mistake travels before something stops it.

In orchestrator/worker, a worker's failure is contained by construction: the orchestrator issued the subtask, so it is the one place that knows the subtask failed, and it can retry, skip, substitute a fallback, or escalate without any other worker being aware. The blast radius is exactly one branch of the fan-out, and the orchestrator's own failure is the only failure mode that touches everything — which is why orchestrator health deserves the tightest monitoring in this topology.

In peer-to-peer, a wrong or stuck agent does not fail in isolation. It has already sent messages to some peers and may be waiting on others. If agent A tells agent B a fact that is wrong, and B acts on it and reports to C, the error has propagated through the mesh before anyone notices — and because no agent has global visibility, "notice" often does not happen until a human reviews the final output. The blast radius is, in the worst case, the entire mesh, bounded only by how many hops the conversation runs before someone applies a check.

In hierarchical systems, blast radius is bounded by branch, which is the whole point of the layering. A failure inside worker.b1 rolls up to supervisor.b, which can contain it — retry, degrade, or report a partial result — without supervisor.a's branch ever being affected. The failure only reaches the root if the owning supervisor cannot resolve it locally, which is a deliberate escalation, not an accident.

Field note: "Blast radius" is a design choice, not a property you discover after an incident. It is set by where you put retry logic, timeouts, and validation — the topology only determines which agents are able to see and contain a given failure.

Latency and cost implications

Topology shapes cost and latency as directly as it shapes failure isolation, because both are functions of how many hops a task takes and how much of that work can run in parallel.

task latency ≈ (hops on the critical path) × (average hop latency) − (parallel savings)
task cost    ≈ (agent invocations) × (average tokens per invocation) × (price per token)

Orchestrator/worker keeps both variables easy to reason about. The orchestrator knows exactly how many workers it dispatched, so cost is close to 1 planning call + N worker calls, and if the subtasks are independent, latency is close to 1 planning call + max(worker latency) + 1 assembly call rather than the sum of every worker's latency. This is the topology's strongest practical advantage: predictable parallelism.

Peer-to-peer cost and latency are both harder to bound ahead of time, because the number of message hops is a function of how the conversation unfolds, not a fixed fan-out the orchestrator decided in advance. Two agents that "almost agree" can exchange several extra rounds before converging, and each round is a full model call. Without an explicit turn limit, a peer-to-peer conversation has no natural stopping point — cost and latency are only bounded by whatever cap you impose from outside the mesh.

Hierarchical systems add a fixed latency cost per layer — each supervisor's summarization step sits on the critical path — but recover some of it by keeping each layer's fan-out narrow enough to parallelize cleanly. A two-layer tree with four workers per supervisor is usually faster in the middle of the distribution than one orchestrator managing sixteen workers directly, because the top-level orchestrator no longer waits on every one of sixteen calls to return before it can summarize; each supervisor summarizes its own four first.

A ballpark comparison

Read reducing LLM costs without hurting quality for the token-level tactics that apply inside any single agent in these topologies — routing to smaller models for narrow subtasks, trimming context, and caching stable retrieval all still apply once you have chosen a coordination shape.

Guardrails and evaluators add a cost that is easy to undercount, because each one runs once per hop, not once per task. An orchestrator/worker system that validates every worker's output before assembly pays for N validation calls on top of N worker calls — a cost that scales the same way the work does, which is usually acceptable. A peer-to-peer mesh that validates every message before it is trusted by the receiving agent pays for one validation call per message round, and because message rounds are not bounded in advance, neither is the validation bill. If you add guardrails to a mesh, budget them against the worst-case number of rounds the conversation is allowed to take, not the typical case — the typical case is exactly what a cost estimate should not be based on when the tail is what causes incidents.

State and context sharing between agents

Every topology has to answer the same question differently: what does an agent need to know that it did not produce itself, and how does that information reach it?

Orchestrator/worker centralizes this decision. The orchestrator decides exactly what context each worker receives — typically a narrow slice of the task plus only the upstream results that worker needs, not the full conversation history. This keeps individual worker context windows small and makes it easy to answer "what did this worker actually see" during a review, because the orchestrator's dispatch call is the single source of truth.

Peer-to-peer pushes that decision onto each agent. Every agent has to decide, per message, how much context to include for its peer, and there is no shared source of truth for "what the mesh currently believes." A common failure mode is context drift: agent A updates its understanding of the task, but B and C are still working from a stale version because nobody pushed the update to them. Some peer-to-peer designs solve this with a shared blackboard — a common store every agent reads from and writes to — which reintroduces a piece of centralized state without reintroducing a coordinator.

Hierarchical systems share context vertically. A supervisor summarizes its branch's outcome before passing it up, which keeps the parent's context small but means the parent only ever sees a summary, never the raw work. That is a deliberate trade-off: it protects the upper layers from context bloat, at the cost of the parent being unable to catch a subtle error that got smoothed over in the summary.

Consider a concrete case: a research agent finds a revised figure partway through a task and updates its own working notes. In orchestrator/worker, that update only matters if the research agent reports it back to the orchestrator, which then decides whether to re-dispatch any worker that already used the old figure — a single, auditable decision. In a peer-to-peer mesh, the research agent would need to know which of its peers already consumed the old figure and message each of them individually; if it misses one, that peer keeps working from stale data with no mechanism to notice. This is the practical shape of context drift, and it is why peer-to-peer designs that expect to run more than a few message rounds usually end up building a shared blackboard specifically to avoid it.

Choose a context-sharing strategy deliberately

Whichever strategy you pick, write down what "current state of the task" means and who is allowed to change it. A multi-agent system with no answer to that question does not have a context-sharing strategy — it has however many strategies its agents individually improvised.

Error propagation and retry semantics

Retries interact with topology in a way that is easy to get wrong: a retry that is safe in one topology can duplicate work or compound an error in another.

In orchestrator/worker, retry ownership is unambiguous — the orchestrator decides whether to retry a failed worker call, and because the orchestrator issued the original request, it can make that retry idempotent (same subtask, fresh attempt) without any other agent needing to know a retry happened at all.

In peer-to-peer, a retry means re-sending a message to a peer that has already, potentially, acted on the first one. If agent A's message to B times out and A retries, but B actually received and processed the original message and is now waiting on a downstream call, A's retry can trigger a duplicate action on B's side unless every message carries an idempotency key that B checks before acting. This is not optional plumbing in a peer-to-peer design — it is close to a prerequisite for a retry to be safe at all.

In hierarchical systems, a retry at a leaf worker is as contained as it is in orchestrator/worker. A retry at a supervisor level is more consequential, because the supervisor is retrying an entire sub-plan, not a single call — which can mean re-dispatching several already-completed workers unless the supervisor tracks partial completion within its own branch.

Decision rule: Only retry a call whose side effects are idempotent, or whose side effects have not happened yet. In a mesh, that means checking with the receiving agent before assuming a retry is safe — you cannot infer it from the sender's own state.

Read AI agent reliability for the general failure taxonomy and bounded-retry model this section builds on; the difference in a multi-agent system is that the "boundary" a retry has to respect is now a boundary between agents, not just between an agent and a tool.

Observability requirements per pattern

What you need to trace to debug a multi-agent failure is a direct consequence of the topology, because the topology determines whether a trace even has a natural root.

Orchestrator/worker produces the cleanest trace shape: one root span for the plan, one child span per worker call, tagged with the subtask it was given and its outcome. Debugging starts at the root and walks down exactly one level to find the failing branch. The minimum you need to record is the orchestrator's dispatch decision (which workers, with what input), each worker's outcome, and the assembly step that combined them.

Peer-to-peer has no natural root, which is the central observability problem of the topology. Without a shared correlation id attached to every message from the first hop onward, a peer-to-peer trace cannot be reconstructed after the fact — you would need to separately query every agent's logs and manually reassemble the conversation in time order. The minimum you need is a task-level correlation id propagated on every message, a sequence number or timestamp per hop, and the sender/receiver pair on each span, so a query across all agents' logs can rebuild one coherent timeline.

Hierarchical traces mirror the tree: a root span for the top supervisor, a child span per branch supervisor, and grandchild spans per worker. This is more spans than orchestrator/worker for the same amount of work, but each span answers a narrower question, and a failure's path to the root tells you exactly which branch and which layer contained it. The minimum you need is a parent-child link at every layer plus each supervisor's summarization input and output, since a bad summary — not just a bad worker call — is a real failure mode here.

What to record regardless of topology

Build a small orchestrator/worker system

The rest of this article works through a minimal orchestrator/worker implementation in TypeScript. It is intentionally small — a planner, three workers, and an assembly step — so every piece is inspectable. Treat it as a starting shape to adapt, not a production framework; a real system would add persistence for in-flight state, a message queue between orchestrator and workers, and the retry and budget checks described above wired into the dispatch loop.

The choice of orchestrator/worker for this example is deliberate, not arbitrary: it is the pattern most teams should reach for first, its blast radius is the easiest to reason about, and its trace shape maps directly onto the code below with no extra translation. If your task turns out to be a genuine negotiation or needs branch-specific policy, the contract, dispatch loop, and observability shape here still transfer — a peer-to-peer version replaces the fan-out with message passing and a correlation id per conversation, and a hierarchical version nests this same dispatch loop inside each supervisor.

Define the contract between orchestrator and workers

Start with the shape of a subtask and a worker result. Keeping this contract explicit is what makes the orchestrator able to retry, skip, or substitute a worker without every worker needing to agree on anything beyond this shape.

// Illustrative: adapt the subtask and result shape to your own worker set.
type SubtaskKind = "search" | "summarize" | "verify";

type Subtask = {
  id: string;
  kind: SubtaskKind;
  input: string;
  timeoutMs: number;
};

type WorkerResult =
  | { id: string; status: "ok"; output: string }
  | { id: string; status: "failed"; error: string }
  | { id: string; status: "timeout" };

type Worker = (task: Subtask) => Promise<WorkerResult>;

Implement bounded dispatch with a concurrency limit

The orchestrator dispatches every subtask in parallel, but bounds concurrency and enforces the per-task timeout itself rather than trusting each worker to respect it. This is the piece that keeps a wide fan-out from turning into an unbounded burst of calls.

// Illustrative: a real implementation would also emit a span per subtask.
async function withTimeout(task: Subtask, worker: Worker): Promise<WorkerResult> {
  const timeout = new Promise<WorkerResult>((resolve) =>
    setTimeout(() => resolve({ id: task.id, status: "timeout" }), task.timeoutMs),
  );
  return Promise.race([worker(task), timeout]);
}

async function dispatch(
  tasks: Subtask[],
  worker: Worker,
  concurrency: number,
): Promise<WorkerResult[]> {
  const results: WorkerResult[] = [];
  let cursor = 0;

  async function runNext(): Promise<void> {
    const index = cursor++;
    if (index >= tasks.length) return;
    results[index] = await withTimeout(tasks[index], worker);
    return runNext();
  }

  await Promise.all(Array.from({ length: Math.min(concurrency, tasks.length) }, runNext));
  return results;
}

Handle worker failure without failing the whole run

A single failed or timed-out worker should not discard the results the other workers already produced. The orchestrator classifies each outcome and decides, per subtask, whether to retry once, substitute a fallback, or mark that branch degraded — then assembles whatever it has.

// Illustrative: bound retries to one attempt per subtask to avoid an unbounded loop.
async function dispatchWithRetry(
  tasks: Subtask[],
  worker: Worker,
  concurrency: number,
): Promise<WorkerResult[]> {
  const firstPass = await dispatch(tasks, worker, concurrency);
  const retryable = tasks.filter((_, i) => firstPass[i].status !== "ok");
  if (retryable.length === 0) return firstPass;

  const retried = await dispatch(retryable, worker, concurrency);
  const retriedById = new Map(retried.map((result) => [result.id, result]));

  return firstPass.map((result) => retriedById.get(result.id) ?? result);
}

function assemble(results: WorkerResult[]): { answer: string; degraded: boolean } {
  const failures = results.filter((result) => result.status !== "ok");
  const usable = results.filter((result): result is Extract<WorkerResult, { status: "ok" }> => result.status === "ok");
  return {
    answer: usable.map((result) => result.output).join("\n"),
    degraded: failures.length > 0,
  };
}

Observe the run with a trace shape that matches the dispatch

The trace should record exactly the decision the orchestrator made, not a generic "agent ran" event. This is the difference between a trace you can debug from and one that only confirms the run happened.

// Illustrative: replace `emitSpan` with your tracing SDK's span API.
async function runOrchestratedTask(tasks: Subtask[], worker: Worker) {
  const runId = crypto.randomUUID();
  emitSpan({ runId, kind: "root", name: "plan_task", subtaskCount: tasks.length });

  const results = await dispatchWithRetry(tasks, worker, 4);
  for (const result of results) {
    emitSpan({ runId, kind: "child", name: `worker.${result.id}`, status: result.status });
  }

  const outcome = assemble(results);
  emitSpan({ runId, kind: "root", name: "assemble", degraded: outcome.degraded });
  return outcome;
}

With this shape, a reviewer opens the root span, sees subtaskCount and the assembly's degraded flag, and only then drills into the specific worker.* child span that failed — the same reading order LLM tracing describes for single-agent spans, applied one level up to a fan-out the orchestrator controls.

Protect the security boundary

Multi-agent coordination multiplies the number of places a permission check can be skipped, because every hop is a candidate place for one agent to trust another agent's claim without verifying it.

In orchestrator/worker, the orchestrator is the natural place to enforce which workers are allowed to take irreversible actions (sending an email, writing a record, spending money) and which are read-only. Keep that boundary in the orchestrator's dispatch logic, not in each worker's own judgment — a worker that decides for itself whether an action is safe is a worker that can be prompted into deciding wrong.

In peer-to-peer, this boundary is much harder to enforce, because there is no single dispatcher checking permissions before a message goes out. If any agent in the mesh can trigger a side-effecting action, every agent that can reach it — directly or through a chain of messages — is effectively part of the trust boundary for that action. Treat any peer-to-peer design with an irreversible action anywhere in the mesh as high risk, and prefer routing that action through a single gatekeeping agent even if the rest of the system stays peer-to-peer.

In hierarchical systems, permission boundaries can follow the branch structure — a billing supervisor's branch can carry stricter approval requirements than a FAQ branch — but only if a parent supervisor actually enforces that a child cannot escalate its own permissions. A worker that can call a tool its supervisor did not authorize defeats the isolation the tree was supposed to provide.

None of the code in this article authorizes an irreversible action; the dispatch example above only assembles text results. A production system needs an explicit approval step, described in AI agent reliability, before any agent in any of these topologies is allowed to take an action a person cannot easily undo.

Roll out a topology change safely

Changing a multi-agent system's topology — splitting one orchestrator into a supervisor tree, or moving two agents from direct messaging to a coordinator — is a bigger change than it looks in a pull request, because it changes where failures are contained and what a trace looks like at the same time. Treat it with the same discipline as any other production change: a baseline, a comparison cohort, and a rollback path.

Run the new topology in shadow mode first where the workflow allows it — let it process the same inputs as the existing system without serving its output to users, and compare the two on completion rate, cost per completed task, and latency. Shadow mode is especially valuable for a peer-to-peer-to-hierarchical migration, because it is the only way to confirm the new tree actually converges to the same answers before you remove the mesh's ability to negotiate its way there.

Stage the rollout by workflow or by a narrow input cohort rather than switching every task over at once. A topology change that improves the median case can still regress the tail — a hierarchical tree's extra hop per layer, for example, can push an already-slow task past a timeout that the flatter orchestrator/worker version never approached. Compare the tail, not just the average, before calling the migration complete.

Keep the previous topology's code path available until the new one has run against a representative volume of real tasks, not just the shadow sample. A topology decision that looked correct on a whiteboard and in a small cohort can still surface a message-ordering bug or a missed retry boundary only under production concurrency — the kind of failure the AI agent production checklist is built to catch before it reaches every user.

Explain what changes the recommendation

The guidance above assumes a task that decomposes cleanly and a team that can instrument every hop. A few conditions change the right answer:

• The task is a genuine negotiation, not a decomposition. Forcing a negotiation into orchestrator/worker produces an orchestrator that adds latency without adding a real decision — peer-to-peer, with a strict turn limit and a convergence check, is the better fit.
• The fan-out is small and stable. A two- or three-worker orchestrator rarely benefits from adding a supervisor layer; the extra hop costs more than the added structure saves.
• You cannot yet observe every hop. If you cannot instrument a peer-to-peer mesh with correlation ids across every agent, do not ship it to a workflow where a wrong answer has real consequences — fall back to orchestrator/worker until the tracing exists, even if the mesh would otherwise be a better conceptual fit.
• A single capable agent already solves the task. Multi-agent coordination is a cost you pay for isolation, parallelism, or specialization. If a single agent's failure rate and latency are already acceptable, adding topology does not improve the outcome.

Make the operating decision

The worked example in this article assembled a plan from independent workers and stopped there. A production version would add a persistence layer for in-flight subtasks, a message queue instead of direct Promise.all calls, and the budget and approval checks described in the retry and security sections above — all without changing the topology decision itself.

Leave the reader with the next move

• [ ] The topology decision is written down, along with the task property that justified it.
• [ ] The blast radius for a single agent's failure is explicit, not implied by the code.
• [ ] Every hop between agents carries a correlation id, an idempotency key where retries are possible, and a terminal status.
• [ ] Irreversible actions are gated by one agent or one policy check, regardless of how many agents can reach them.
• [ ] The rendered article, including the topology explorer and both diagrams, has been checked at desktop and mobile widths.

Start with the smallest version of the topology you chose — one orchestrator and two or three workers, or one supervisor and a narrow branch — and instrument it fully before adding a second layer or a fourth peer. A multi-agent system earns more agents by proving the coordination boundary works, not by adding capacity in advance of a need you have not yet observed.

Frequently asked questions