Designing the agent team: roles, scope and handoffs

The design happens before the prompt

Most failed agent projects were lost at the whiteboard, not in the code. Someone picked a framework, wired up four agents that sounded sensible, and discovered three months later that nobody could say which agent was responsible for a wrong result. An agent team is an organisational design problem first: roles, authority, handoffs and escalation. The prompts come last, and they get much easier once the roles are right.

This article describes the method we use in the design phase of a build: how a real process becomes a set of agent roles with boundaries you can test and audit.

Start from the process, not the org chart

Map the process as it is actually performed, with the people who perform it, step by step. For each step, record four things:

• Input: what arrives, in what form, from where.
• Decision: what judgment the step requires, if any.
• Effect: what changes in the world when the step completes.
• Reversibility: how expensive it is to undo that effect.

Do not model the departments. A process that crosses finance and procurement does not need a "finance agent" and a "procurement agent" because the company has those two teams. It needs agents shaped like the work: a document agent that reads what arrived, a matching agent that reconciles it against a record, a posting agent that writes to a system of record.

What makes a good agent role

A role is well drawn when you can write a single sentence of the form "this agent turns X into Y, and may touch nothing else." Four properties follow from that:

• One job. If the role description contains "and also", split it.
• Bounded input. The agent receives a defined payload, not "the conversation so far".
• Explicit authority. The tools it may call are listed, and nothing else is reachable.
• A stopping condition. The agent knows what "done" looks like and what to do when it cannot get there.

Roles that violate these are usually too large. An agent asked to "handle vendor onboarding end to end" will improvise; three agents with narrow jobs and an orchestrator that sequences them will not.

The authority table

For every agent, fill in one row before any implementation starts. This table becomes the permission model, the test plan and the thing you show a security reviewer.

Agent	May read	May change	Decides	Must escalate when
Documents	Inbox folder, document store	Nothing	Document type, extracted fields	Confidence below threshold, unreadable file
Matching	Extracted fields, purchase orders	Nothing	Match / no match, discrepancy list	Partial match, price variance over tolerance
Posting	Matched record	ERP: create draft, post	Nothing	Always, above the approval threshold

Two rules make this table useful rather than decorative. First, permissions belong to tools, not to agents: the posting agent cannot post because the model decided to, it can post because the tool it is allowed to call performs exactly that operation and nothing wider. Second, read and write are separate tools, ideally separate servers, so a read-only agent cannot be talked into writing.

Handoffs are contracts, not conversations

The weakest point in most multi-agent systems is the handoff. Agents pass each other free text, the next agent re-interprets it, and small misunderstandings compound. Treat every handoff as a typed payload with a schema: what fields, what types, what is required, what a failure looks like.

In a hub-and-spoke team the handoff is always agent to orchestrator and back, never agent to agent. That single rule removes an entire class of debugging: when the output is wrong, the orchestrator's log shows exactly which payload entered which agent and what came out.

A good handoff payload also carries provenance: which document, which record id, which confidence, which model produced it. When a human reviews the result later, they should not have to re-derive where a number came from.

Where the team ends and a person begins

Every design has a line past which the system does not act alone. Draw it with the reversibility column from your process map, not with intuition. Steps that are cheap to undo (drafting, classifying, gathering) can run unattended. Steps that move money, contact a customer, or create a record other systems depend on wait for a person.

Two distinct mechanisms are involved, and they are often confused. An approval gate is planned: the system knows in advance that this step requires a signature, and the work stops there by design. A delegation to a human is dynamic: the agent has hit a case it cannot resolve and hands it to a person mid-flight. The first is covered in human-in-the-loop design; the second is a capability the team needs from day one, described in delegating to a human.

How many agents?

Fewer than people expect. A useful heuristic: one agent per distinct skill set in the process, not one per step. If two steps require the same knowledge and the same tools, they belong to the same agent. If a step needs different context, different tools or a different risk level, split it out.

Four to six specialist agents covers most back-office processes. Beyond that, ask whether you have discovered a second process wearing the first one's clothes, and whether it deserves its own team.

Test the design before you build it

You can evaluate a role design on paper. Take ten real cases from the last month, including the three that went wrong, and walk them through the roles you drew:

• Does every case reach a terminal state, or do some fall between two agents?
• Can you name, for each case, the single agent accountable for the outcome?
• Does any agent need data it has no permission to read?
• Which cases hit an escalation, and does a person have enough context to act?

Cases that stall in this exercise will stall in production. Redraw the roles now, while the design is a table and not a codebase.

What you take into the build

The output of a design phase is small and concrete: a process map, an authority table, a handoff schema per edge, a list of escalation conditions, and ten test cases with known correct outcomes. With those in hand, building the team becomes an ordinary engineering task, and the first evaluation run has something to measure against.

Frequently asked questions

How long does designing an agent team take?

For one process, usually one to two weeks of part-time work with the people who run it. The output is a process map, an authority table per agent, handoff schemas and a test set. That artefact is what makes the build predictable, and it is worth keeping even if you decide not to build.

Should each agent have its own model?

Not necessarily its own model, but usually its own model choice. The orchestrator benefits from a stronger model because it plans and verifies; narrow specialists often run well on smaller, cheaper ones. Decide per role against the evaluation set rather than standardising on one model everywhere.

What if the process is not documented anywhere?

That is the normal case. The design phase doubles as documentation: mapping the process with the people who perform it usually surfaces exceptions and informal rules that were never written down, and those exceptions are exactly what determines the agent boundaries.