A vote does not make answers independent
The easiest reliability strategy sounds persuasive: run several agents and accept the majority answer. It works only when their errors are sufficiently independent. Production assemblies often use the same model, shared context, identical sources, and one reasoning pattern. Voting then reproduces a common failure.
Agents Catching Agents tested this effect on clinical tasks. One false suggestion changed an individual agent's decision in 5% to 16% of cases. When two apparently independent participants gave the same wrong answer, a third accepted it 38% of the time.
A conventional oversight gate performed poorly, reaching 100% false positives in one setting. A referee that solved the task privately, without seeing the committee discussion, worked better. On image tasks it achieved 77% to 88% precision at a 13% to 21% false-positive rate.
The data are clinical, so the exact rates do not transfer directly to software engineering or research. The mechanism is broader. A verifier that reads three confident answers from similar agents has not received three pieces of evidence. It has received three forms of the same influence.
Coordinators can lose the fact that mattered
Correct worker outputs can still be damaged during handoff. One agent finds a constraint, another builds a plan, and a third writes code. The coordinator compresses results to fit the context and removes the condition that controls the next branch.
OrchBench separates orchestration quality from worker quality. It represents a task as a dependency graph. The planner assigns subtasks and decides which results move downstream and at what fidelity. A deterministic simulator estimates quality, token cost, and makespan without running real workers.
Simulator scores correlated with Claude Code executions at 0.816 while using 1.3% of the tokens and 10.3% of the time. The central result was not about adding agents. Preserving critical information mattered more than increasing the worker count.
Simulation cannot replace a final execution. It may model a rare transition or unexpected tool behavior incorrectly. It can cheaply screen plan structures and reveal where one branch's output disappears before use.
Shared context creates a shared source of error
Many multi-agent systems are distributed only in the diagram. Every participant reads the same summary, uses the same search results, and calls models from one family. A plausible error in the initial material is then reinforced by each subsequent rewrite.
Independence has to be designed. A verifier should not see the conclusion before making its own attempt. Research agents need sources with distinct provenance rather than ten summaries of one announcement. A code change needs executable tests, not a second agent's opinion that the patch looks reasonable.
Method diversity matters too. One participant may derive an answer from text, another recompute it from data, and a third inspect the resulting external state. Three models continuing the same chain of thought do not provide that diversity.
Handoff state is part of the deliverable
Every dependency in a plan needs an explicit contract: what is transferred, in which representation, and how the consumer checks completeness. A conclusion alone is not enough. The handoff should preserve inputs, constraints, evidence links, unresolved questions, and confidence.
Context compression also needs provenance. If the coordinator drops part of a result, it should retain a pointer to the source that can be reopened. Otherwise a short summary becomes irreversible data loss.
A useful orchestration metric is not the number of messages. It is the share of dependencies for which the downstream step received all critical inputs. Another is the amount of repeated research caused by losing a fact that had already been found.
This can be audited without an expensive model fleet. Reconstruct the dependency graph from completed traces, mark where each required fact first appeared, and inspect what reached its consumer. A fact lost during summarization is a handoff failure, not a reasoning failure.
When another agent actually helps
An additional participant is valuable when it contributes a different observation or verification method. One agent may propose a change, another execute isolated tests, and a third compare actual external state with predeclared success conditions.
For a disputed decision, the referee should work privately first. Only after producing an answer should it receive the group's conclusion and evidence. A disagreement should trigger diagnosis, not another vote.
When every participant uses the same model and context, it is more accurate to call them repeated attempts by one agent. Repetition can improve the chance of a successful sample. It does not create independent confirmation.
Agent count is a cost parameter. Reliability comes from independent evidence, different verification methods, and lossless state transfer.