A multi-agent demonstration can make a difficult workflow look almost effortless. A request enters the system, specialist agents divide the work, and a polished result appears.
Production is less forgiving.
Real systems encounter partial failures, stale context, conflicting decisions, unavailable tools, cost limits, and actions that cannot safely be reversed. Once agents can modify data or trigger external operations, the quality of the model is only one part of the design.
The central challenge is familiar: coordination under uncertainty.
The three failure modes I design for first
Most fragile agent systems fail in one or more of three predictable ways.
1. Coordination failure
Agents duplicate work, operate on different assumptions, or complete tasks in the wrong order. The system has activity but no dependable source of truth.
2. State failure
Important decisions disappear between invocations. An agent repeats work, contradicts an earlier choice, or cannot explain how the current result was reached.
3. Visibility failure
The final output is wrong, but the team cannot reconstruct which agent made the incorrect decision, which tool result it used, or what context was available at the time.
A more capable model may reduce the frequency of individual mistakes. It does not remove these structural risks. They require architecture.
Start with the workflow, not the number of agents
Before introducing multiple agents, I define the workflow as a task graph:
Each node should have an owner, inputs, outputs, dependencies, status, and completion criteria. Whether a node is executed by an agent, a deterministic service, or a person is a secondary decision.
This prevents the architecture from becoming a collection of agent personalities without a coherent operating model.
Layer 1: Orchestration
The orchestrator is responsible for turning a goal into controlled work. It routes tasks, tracks dependencies, applies retry policy, and decides when human input is required.
Three patterns are common:
Hub and spoke. One orchestrator delegates bounded tasks to specialist workers. This is easy to understand and works well for workflows with a clear central decision-maker.
Hierarchical delegation. An orchestrator delegates a larger area to another coordinator. This can help with complex programmes of work, but it increases the amount of shared state and failure handling.
Parallel worker pool. Independent items are dispatched to equivalent workers and aggregated. This is effective for batch analysis, classification, or other naturally parallel tasks.
The pattern matters less than the contract. The orchestrator needs a persistent task graph and an idempotent way to update it. If a worker retries, the system must know whether to resume, replace, or ignore the previous attempt.
Layer 2: State and memory
Agent memory is often discussed as if it were a single feature. In production, it is better treated as several forms of state with different lifetimes.
| State type | Purpose | Typical storage |
|---|---|---|
| Working state | Current task, intermediate results, active constraints | Structured task context |
| Execution history | Actions, tool calls, outcomes, approvals | Transactional event log |
| Domain knowledge | Documentation, schemas, policies, code | Searchable knowledge source |
| Learned preference | Reviewed patterns and accepted decisions | Curated records |
The minimum useful implementation is not an unlimited conversation history. It is a structured task record and an append-only execution log.
Unfiltered history can be actively harmful. It consumes context, preserves outdated assumptions, and makes relevant evidence harder to find. Retrieval should be scoped to the current task and should expose where each piece of context came from.
Layer 3: Tool boundaries
An agent becomes operationally significant when it can call tools. That is where permissions and validation have to become explicit.
MCP provides a useful standard interface:
A production tool should define:
- A narrow purpose
- A typed input schema
- Server-side validation
- The identity and permissions used for execution
- Timeout and retry behavior
- A structured result
- An audit record
The tool should also be safe to call more than once whenever possible. If an operation is not idempotent, the orchestrator needs a request identifier or another mechanism to prevent duplicate side effects.
Broad tools transfer too much policy to the model. A focused business operation such as create_draft_invoice is easier to secure and observe than unrestricted database or shell access.
Layer 4: Evaluation
Agents produce probabilistic output, but production acceptance criteria do not need to be probabilistic.
For a coding workflow, evaluation may include:
- Compilation
- Unit and integration tests
- Static analysis
- Security scanning
- Contract or snapshot tests
- A review against the original requirements
For research or content workflows, evaluation may include:
- Source coverage
- Citation validity
- Required topic coverage
- Detection of unsupported claims
- Human review for tone and judgment
I keep evaluation separate from generation. The same agent that produced an answer should not be the only mechanism deciding whether that answer is acceptable.
Regression suites are especially important. Model, prompt, and tool changes can alter behavior in ways that are difficult to notice from a few successful examples. A representative evaluation dataset gives the team a basis for comparison before deployment.
Layer 5: Observability and governance
When an agent system fails, a team should be able to answer:
- What goal was the system pursuing?
- Which agent made each decision?
- What context and tool results were available?
- Which model and prompt version were used?
- What did the action cost?
- Was a human approval required and received?
- Can the execution be replayed or investigated?
This requires structured telemetry rather than a collection of chat transcripts.
A useful event schema might include:
Logs must protect sensitive content, but removing all detail makes them useless. The design needs deliberate redaction, retention, and access policies.
Human approval is part of the architecture
Some actions should never be inferred from a broad objective.
Deleting records, deploying to production, sending external messages, changing access permissions, and committing financial transactions are examples of operations that usually deserve an explicit approval boundary.
That boundary should live in the workflow engine or tool layer, not only in the prompt. A prompt is guidance. An enforced permission check is a control.
The user should see the exact proposed action, relevant consequences, and scope before approving it. After approval, the system should record who approved what and which execution consumed that approval.
Cost and time need hard limits
Agent loops can continue making locally reasonable decisions while producing little additional value. Production systems need budgets:
- Maximum model calls per task
- Maximum elapsed time
- Token or financial budget
- Retry limits per tool
- Concurrency limits
- Circuit breakers for repeated failure
These limits should produce an explicit terminal state. Silent truncation leaves operators unable to distinguish completion from exhaustion.
Why I usually start with one agent
The most reliable route to a multi-agent system is often to avoid starting with one.
I begin with a single agent, a small toolset, and clear evaluations. That reveals the actual constraints:
- The context may be too large for one worker
- Independent tasks may justify parallel execution
- A separate reviewer may improve quality
- Different permissions may require isolated roles
- Latency may justify specialist paths
Only then do I add agents to address a measured limitation.
This keeps the system understandable. It also gives each new agent a specific responsibility and a reason to exist.
Production readiness checklist
Before treating a multi-agent workflow as production-ready, I expect:
- A persistent task graph with explicit dependencies
- Structured records for every agent and tool action
- Idempotency or duplicate protection for side effects
- Narrow, authenticated tool contracts
- Evaluations that run when models, prompts, or tools change
- Human approval for irreversible or external actions
- Cost, time, retry, and concurrency limits
- Defined recovery behavior for partial failure
- A way to reconstruct an execution during an incident
- Clear ownership for operating the system
None of these controls are unique to AI. They come from distributed systems, security engineering, and production operations.
That is the main point. Multi-agent systems introduce a new type of worker, but they do not remove the need for established engineering discipline. The teams that build dependable agent systems will be the ones that treat orchestration, state, permissions, evaluation, and observability as first-class product capabilities.
