Choosing between one generalist and a team of specialists is a familiar management decision. AI architecture raises a similar question.
A capable generalist runs everything: one person, full context, fast turns between thinking, asking, checking, deciding. That works until the workload exceeds what one person can handle. Then you do what humans have always done. You split the work, hire specialists, and accept the cost of coordination — the meetings, the handoffs, the slack messages clarifying what the email meant.
An AI architecture raises the same trade-off, even when the decision arrives under the name “model selection” or “agent strategy.” Describing the work first gives your team a way to evaluate the vendor’s recommendation.
You Have Already Made This Decision
I work with AI systems every day, and the pattern I keep noticing is that frontier models are no longer behaving like tools. They are behaving like operating models for cognition. One path leans toward a single very capable brain — unified context, fast transitions between reasoning, tool use, checking, and continuing. Another leans toward a team of specialists — agents that take a slice of the work, hand it off, and report back. Both are real, both are in production, and neither is universally better.
What is new is the scale. McKinsey’s 2026 State of Organizations surveyed more than 10,000 senior executives across 15 countries and 16 industries and found organizations shifting from short-term resilience toward sustained productivity with AI at the core of transformation. The World Economic Forum’s 2026 report puts the shift bluntly: from isolated use cases to connected systems, from episodic initiatives to continuous processes that “sense signals, make decisions” and coordinate execution across functions.
The more your work runs on AI, the more you have to decide where intelligence lives. Centralized in a shared platform? Distributed across local teams? Some hybrid?
A recent academic paper calls AI “coordination-compressing capital.” The idea is that AI changes the effort of keeping work aligned as well as the effort of doing each task. It affects who needs to coordinate with whom, how often, and at what cost.
Adoption is no longer the question. Stanford HAI’s 2026 AI Index puts organizational adoption at 88%. The bottleneck has moved.
Two Architectures, One Tradeoff
The better architecture depends on how the work can be divided.
A 2025 study by Google Research and MIT tested 180 configurations across five agent architectures and three LLM families, varying centralization, decomposition and coordination. Results differed by task type.
On parallelizable tasks like financial reasoning — work that divides into independent subtasks — centralized multi-agent coordination improved performance by 80.9% over a single agent. On sequential reasoning tasks — work that needs a coherent thread from start to finish — every multi-agent variant degraded performance by 39 to 70 percent.
The same multi-agent approach helped on parallel work and hurt on sequential work. Task structure changed the result.
The organizational-design connection is that team structure follows work structure. Independent tasks and a continuous line of reasoning need different arrangements. Galbraith would recognize it. So would anyone who has run a sequential analysis through a committee and ended up with a worse answer than a single thoughtful person.
Microsoft’s Cloud Adoption Framework offers a usable rule of thumb. For workflows under three to five distinct functions, use a single agent — consolidating logic, context, and tool execution reduces complexity and simplifies governance. For solutions spanning more than that, decompose into specialists. If you have ever debated whether a new team needs its own department or can stay inside an existing one, the threshold will feel oddly familiar.
Stanford HAI’s AI Index also reports model capability climbing fast — SWE-bench Verified performance rose from 60% to near 100% in a single year. Capability is no longer the bottleneck. Design is. The architecture question gets sharper as the models get better, not softer.
The Coordination Tax
A 2025 practitioner analysis in Towards Data Science documented what happens when teams decompose AI work without measuring the cost. A task that costs ten cents with a single agent costs a dollar fifty with a multi-agent system — a 15x multiplier. Every additional agent demands shared context, handoff protocols, and conflict resolution. Its contribution has to justify the work of keeping it aligned with the others. Brooks’s Law in new clothing: adding more agents to complex reasoning makes it slower, more expensive, and less coherent.
The Google/MIT study quantified the upper bound. Coordination gains plateau beyond four agents. Additional agents added overhead without improving throughput. On strict sequential reasoning, communication overhead fragments the process — agents spend so much of their context window negotiating with each other that they have insufficient cognitive budget for the actual work. Anyone who has watched a team spend more time in alignment meetings than doing the work knows the shape of that failure.
Deloitte’s 2026 State of AI in the Enterprise found only one in five companies has a mature governance model for autonomous AI agents, even as agentic usage rises sharply. The overhead is real, the budget is unset, and most organizations will not see the problem until the invoice arrives.
The Google/MIT researchers could predict optimal architecture from task structure with 87% accuracy. The work answers the architecture question more reliably than the model does.
The Org Chart Inside the Model
Since early 2025, nearly all leading frontier AI models have converged on Mixture of Experts architecture. Dense networks — where every parameter activates for every input — are the AI equivalent of an organization where every employee works on every project. They scale until they do not. Activating all parameters for every query became unsustainable around the hundreds-of-billions-of-parameters mark.
MoE architectures route each input through a small subset of specialized “expert” sub-networks. A gating mechanism decides which experts are relevant — the way a project manager pulls specialists into a meeting. Only two to four experts (of eight to sixteen or more) activate per token. Comparable or superior performance at a fraction of the inference cost.
The specialist team is not a metaphor for the model. It is the model.
That explains something benchmarks were never going to capture. Bigger generalist models break down when work moves into specialized professional domains. A senior tax specialist will outperform a brilliant generalist on a corporate restructuring question every time. The same logic applies to an AI tuned for those domains versus a general-purpose model trying to handle everything.
The portfolio approach has quietly become the practitioner default: generalists for exploration, domain-specialized models for high-stakes decisions, coordination agents for the seams. Mature AI strategy looks suspiciously like mature talent strategy. You hire generalists and specialists, you do not pretend either alone is enough, and you spend serious time on the handoffs.
Is the Metaphor Doing Real Work?
The comparison with human teams is useful, but it needs limits.
The first: the organizational design metaphor is misleading. The risk is that decision-makers import assumptions from human org design that do not transfer — a CEO who prefers flat organizations chooses distributed multi-agent systems even when a centralized model would perform better, because the metaphor activates an existing mental model. Fair concern. Analogies that feel right are exactly the analogies that bypass scrutiny.
The comparison rests on measured behavior: task structure changed which architecture performed best, coordination gains plateaued at four agents, and communication overhead disrupted sequential reasoning. The metaphor is useful because it opens the right questions — where to centralize, where to decompose, how much overhead to tolerate. The decision itself stays empirical.
The second: real deployments are hybrid, so “one brain or specialists” oversimplifies. True — the framing is a starting point, not a stopping point. MoE proves the spectrum lives inside a single model, and Microsoft’s three-to-five function threshold gives the nuance. The comparison makes the tradeoff visible before you design a hybrid. That helps move model choice beyond a single bake-off score, which is still how most executives approach it.
The third is the hardest. If multi-agent coordination can create a 15x cost multiplier and 39-70% degradation on sequential tasks, specialist systems are impractical for most organizations today. However, the same study showed 80.9% improvement on parallelizable tasks. The point is not that specialists always win. The point is that the right architecture depends on the work — and most organizations have never asked the question. Deloitte’s one-in-five governance finding tells you where the gap is: autonomy is being deployed faster than architecture is being designed.
Designing Your AI Org Chart
I use four steps to compare architectures against the work they need to do.
Diagnose work structure. Classify the work on two dimensions: parallelizability (can subtasks run independently?) and reasoning continuity (does the task need a coherent thread from start to finish?). High-parallel / low-continuity favors specialists. Low-parallel / high-continuity favors one capable model. High-parallel / high-continuity needs careful orchestration. Low-parallel / low-continuity — any architecture works; pick the cheapest. The most common failure is skipping diagnosis and choosing the architecture that is technically exciting rather than fit-for-purpose.
Match architecture to work. Below three to five distinct functions, default to single-agent. Above that, decompose. For workflows needing both sequential reasoning and specialized knowledge, build hybrid: a central coordinator with specialist tool calls. Always test against a single-agent baseline before scaling. Harvard Business School’s study of 791 product developers at Procter & Gamble found AI-enabled teams produced top-10% ideas three times more often than individuals without AI, with 13-16% time reductions — but the lift was strongest when AI was embedded in a disciplined team structure, not when it replaced structure.
Set the coordination overhead budget. Cap multi-agent cost at roughly 3x the single-agent equivalent on the same task. If coordination tokens consume more than 40% of context, fall back to single-agent — the agents have insufficient cognitive budget for the real work. Without an explicit budget, there is no trigger to reassess, and the system degrades gradually until the invoice makes the architecture choice for you in retrospect.
Measure and shift. Treat the architecture as a choice to test again as the work and models change. Run quarterly reviews. As frontier models improve, yesterday’s specialist team may be outperformed by today’s generalist model. Track three signals: coordination overhead trending upward (too distributed), single-agent performance closing in on multi-agent on the same tasks (consolidate), new task types not fitting current architecture (expand). The General Atlantic 2026 analysis describes operating models where two to five humans supervise 50 to 100 specialist agents across operations, product, finance, and expansion. The leverage is real. It is also a moving target.
One more dimension. The same coordination-compressing AI produces broadly distributed gains or superstar concentration depending on who has access. Architecture is not just an internal design question. Who in your organization gets to design AI org charts? Is that the right table?
I wrote about the human side of this question in The Org Chart After AI. This essay is the AI side. They are converging.
What I’d Do This Quarter
Pick one of your AI deployments and ask a single question: did you choose the architecture because the work demanded it, or because the vendor offered it?
Most teams I talk to followed the vendor recommendation without testing alternatives. Measure token usage, latency, and cost of multi-agent versus single-agent on the same representative task. Force decomposition to earn its place. Put architecture review into AI governance the same way you put it into people governance — every quarter, not when something breaks.
Give both arrangements the same representative task and include the coordination cost in the comparison. A team should be able to explain what the extra agents contribute before taking on the work of keeping them aligned.
References
- Google Research & MIT (2025). “Towards a Science of Scaling Agent Systems.” https://research.google/blog/towards-a-science-of-scaling-agent-systems-when-and-why-agent-systems-work/
- NVIDIA (2025). “Mixture of Experts Powers Frontier AI Models.” NVIDIA Blog. https://blogs.nvidia.com/blog/mixture-of-experts-frontier-models/
- Towards Data Science (2025). “The Multi-Agent Trap.” https://towardsdatascience.com/the-multi-agent-trap/
- Harvard Business Review (2025). “Should Your Business Use a Generalist or Specialized AI Model?” https://hbr.org/2025/07/should-your-business-use-a-generalist-or-specialized-ai-model
- arXiv (2026). “AI as Coordination-Compressing Capital.” https://arxiv.org/abs/2602.16078
- Microsoft (2025). “Choosing Between Single-Agent and Multi-Agent Systems.” Cloud Adoption Framework. https://learn.microsoft.com/en-us/azure/cloud-adoption-framework/ai-agents/single-agent-multiple-agents
- Foster Fletcher (2025). “Enterprise AI Strategy: Why One LLM Is a Costly Mistake.” https://fosterfletcher.com/specialist-ai-year-2026/
- McKinsey & Company (2026). “The State of Organizations 2026.” https://www.mckinsey.com/capabilities/people-and-organizational-performance/our-insights/the-state-of-organizations
- World Economic Forum (2026). “Organizational Transformation in the Age of AI.” https://reports.weforum.org/docs/WEF_Organizational_Transformation_in_the_Age_of_AI_How_Organizations_Maximize_AI’s_Potential_2026.pdf
- Deloitte (2026). “The State of AI in the Enterprise 2026.” https://www.deloitte.com/us/en/what-we-do/capabilities/applied-artificial-intelligence/content/state-of-ai-in-the-enterprise.html
- Stanford Institute for Human-Centered AI (2026). “AI Index 2026 Report.” https://hai.stanford.edu/ai-index/2026-ai-index-report
- Harvard Business School Working Knowledge (2024). “When AI Joins the Team, Better Ideas Surface.” https://www.library.hbs.edu/working-knowledge/when-ai-joins-the-team-better-ideas-surface
- General Atlantic (2026). “AI Is Killing the Century-Old Organizational Pyramid.” https://www.generalatlantic.com/insights/ai-is-killing-the-century-old-organizational-pyramid/