One agent or several: how to decide the architecture of an agentic system

In short. How many agents to put in an AI system is a design decision with consequences for cost, permissions and accountability, and it should be taken like any other investment decision: with a measured baseline and an alternative ruled out in writing. In the ZeroFive.AI method the baseline is a single agent, with the tools the process requires and constrained planning, tested on a set of real cases. A system is split into several agents where part of the process has to be governed differently, and only if the split version beats the baseline on the same budget. In every configuration the sequence of steps and the writes to company systems stay with deterministic code, and the harness is designed before the agents.

The diagrams that come with proposals for agentic projects almost always show a team: one agent that plans, one that searches, one that writes, one that reviews, sometimes a supervisor coordinating them all. The drawing is persuasive because it mirrors an org chart, and anyone looking at it in a management committee recognises the way an office works, with its handovers and its reviews.

We went through this ourselves while designing the agents we use internally to manage projects and update our own software: the first version drawn on paper had more agents than it needed, and the system improved as we removed them. That experience shapes how we approach the question with clients today, which is the same way we treat every other choice in an AI initiative, inside the decision gates of our method.

The number of agents as a design decision

Every additional agent is a component with a model to pay for at every call, permissions to assign, a log to read and a person who answers for it. Multiplying agents multiplies all of these items, and the benefit has to be demonstrated with the arithmetic used for any expense, that is, against the simplest option that solves the same problem.

This is why the question of how many agents are needed comes last, at the top of a ladder of configurations climbed one rung at a time, and only when the rung below has been measured and falls short.

The first rung coincides with G0: a use case enters the portfolio only after the non-AI alternative has been assessed and ruled out in writing, and in a share of cases it is the alternative that wins, at a fraction of the cost of an agent. The second and third rungs match the levels of autonomy we described when writing about skills and agentic planning. The fourth is the subject of this article, and it should be reached with the numbers from the first three already in hand.

The single-agent baseline

The baseline is built with an internal evaluation set, the same tool we use to choose the model for the task: between thirty and fifty cases taken from the real process, each with the outcome an experienced professional would consider correct, including the cases where the right outcome is to stop because the information is missing. A single agent runs on that set, with the tools the process needs and planning limited to the declared skills.

The test produces the numbers that follow the project all the way into operation. The share of correct outcomes tells you whether the system does the job, and the cost per correct outcome tells you what it costs to do it well; it matters more than the cost per run, because a wrong run is paid for twice, once to produce it and once to notice it. Alongside these sits the share of cases in which the agent stops when it should, which in regulated processes is worth as much as accuracy.

When the baseline stays below the threshold, the cause has to be found before components are added. If the model gets it wrong even with the right context in front of it, the limit is its capability, and the fix is a different model or a narrower task. If tools or reliable data are missing, the work lies in the lower layers described in the eight-layer architecture, and no additional agent replaces it. Only when different parts of the process need contexts too large or too heterogeneous to fit in a single run does splitting become an option, and even then it has to be justified.

The criteria for splitting a system

The criterion we apply in projects follows the boundaries of governance: a system is split where something changes that has to be governed differently, and kept whole where the only thing that changes is the role described in the model's instructions.

The first boundary is the class of data. In our method every piece of data belongs to a class (open, internal or confidential) that determines which models may read it, and a process that touches confidential data in one step and public information in another is better designed with two agents: the one working on confidential data runs on a model inside the company perimeter, the other can use the most capable model on the market. With a single agent the whole context would travel to the same model, and the choice would be between paying for the internal model even for trivial work or sending out what should stay in.

The second boundary is write permissions. An agent that reads documents and one that books an order in the ERP have different risk profiles, and keeping them apart limits what an error can reach. The rule that follows fits in one line: whoever gathers and proposes does not write, and writes go through a single deterministic component that applies the approval thresholds. It is the boundary we described in probabilistic and deterministic, where the model interprets and proposes while code calculates and decides.

The third boundary is accountability. When a process crosses two functions with two people in charge, for example the commercial review and the credit check, two agents let each named owner answer for their own perimeter, stop it and read its log without depending on the other function.

The role written into the prompt, by contrast, does not justify a split. A "writer" and a "reviewer" that use the same model on the same data with the same tools produce a second inference pass over the same reasoning, and the review that is actually needed is deterministic where a rule can be written (a total, a format, a threshold) and human where it cannot.

The harness before the agents

In the article on the agentic company we described the four families of rules we use to write the harness: three-level tool permissions, human approval of writes, deterministic checks, an action log. In a multi-agent system the harness has one more job, coordination, and here we apply a few design rules that hold whatever framework is chosen.

The sequence of steps is a plan the system produces as data before executing it, and it stays in the log next to what actually happened, so that intention can be compared with execution. Code executes the plan: code starts the agents, passes results from one to the next, applies timeouts and stop conditions, and no model decides on its own when the work is finished. Deterministic checks run between one step and the next, because an error stopped where it originates costs a fraction of one that reaches the end of the chain looking like a confident answer.

Agents exchange objects with a declared structure, taken from the enterprise ontology, instead of free text to be reinterpreted. A "customer" passed from one agent to another has the same fields and the same meaning for both, and this removes a whole family of errors that show up in testing as misunderstandings between components. Tools reach the agents through MCP servers that operate with the real permissions of the person the system is working for, never with a technical account whose perimeter has no limits.

The harness is also where design meets regulation. For high-risk systems, Regulation (EU) 2024/1689 (the AI Act) requires automatic event logging in Article 12 and human oversight in Article 14, obligations that for Annex III systems, after the Digital Omnibus, apply from 2 December 2027. In a multi-agent system the log has to make it possible to reconstruct the whole chain, who proposed what, which component wrote and who approved, and not only the behaviour of each agent taken on its own. A harness built this way produces much of the evidence an ISO/IEC 42001 management system also needs, and prepares the organisation for the checks that remain the job of lawyers and accredited certification bodies. In our training programmes this is the discipline we call harness engineering, and we put it before any discussion of models and frameworks.

The measures before release

A multi-agent system goes into production with a small dashboard of measures, calculated on the same evaluation set as the baseline and repeated at every release.

The last three rows are the ones most often missing in the projects we see, because no demo puts them on display. They weigh as much as the first ones in the decision to switch the system on, and once it is running they have to be recalculated every time a vendor updates the model, since an update can change an agent's behaviour without anyone touching the code.

The gates for a multi-agent system

The six gates of the method apply to any AI initiative. For a multi-agent system each one carries a specific criterion, non-compensable like the others.

G2 is the gate where a multi-agent architecture is most often scaled down, and that outcome is as useful as a green light, because it arrives before the company has paid for integration with the real systems. Validation is done with fast prototypes on real data in our protot.ai unit, with the hypothesis written down before the result is known. The G3 delegation perimeter is declared agent by agent using the AI-assisted, AI-first and AI-native levels, and two agents in the same system can legitimately sit at different levels.

The most frequent design mistakes

The most common mistake is designing the system from the org chart, with one agent for every professional role involved in the process. The result looks like the office it is meant to assist and behaves like a single agent talking to itself, at the cost of four or five.

Almost as frequent is distributing the writes: each agent updates the systems it touches, and conflicts between two versions of the same record are discovered at the end, when the chain has already produced its effects. Centralising writes in a deterministic component costs little at design time and a great deal to add later.

Then there is the architecture chosen from a framework or a vendor demo, which arrives with a number of agents already decided by whoever built it to show off the product. A company that starts there ends up justifying after the fact a structure nobody measured, while one that starts from the baseline chooses the framework afterwards, knowing what it needs from it.

Finally there are systems switched on without an inventory. For logging and accountability purposes each agent counts as an AI system, and a five-agent system means five entries in the AI system inventory, with five owners, five perimeters and one log holding them together.

Where to start

The work begins with the baseline: one process, a set of real cases, a single agent and its numbers. With those numbers the decision on how many agents are needed becomes a documented decision that can be presented to a risk committee and defended in front of an auditor, and quite often it turns out that the right system is smaller than the one drawn at the start.

The framework of delegation levels and perimeter is in four layers and one perimeter, the full map of the stack is in the agentic company in eight layers, and the service through which we design agents and take them into operation is described on the AI Agents page. To find out where your organisation stands before switching on its first agent, you can start an AI Rating or book an assessment call.