A lot of model selection still happens as if intelligence were one-dimensional. Pick the strongest model, deploy it everywhere, optimize later. The leaderboard says this one is smartest, so it goes on every workflow, and any uneven results get filed under “we just need to prompt it better.”
I have watched teams apply this approach across very different kinds of work. They take the model that tops the benchmark and run it through customer triage, contract review, financial analysis, and code generation — the whole portfolio, one model, every job — then find that one goes beautifully while another quietly falls apart.
Here is the part the leaderboard never shows you. The same model that scores 84-89% on synthetic coding benchmarks scores only 25-34% on real-world, class-level software tasks [1]. Same model. Same skills. A fifty-to-sixty point collapse, from “nearly aced it” to “barely showed up.” That gap is not a rounding error, and it is not a prompting problem. It is the cost of asking the wrong question.
Because the question was never which model is best in general. The question is: what is the shape of the work?
Model Selection Still Treats Intelligence as One Number
Benchmarks give you a comparable number, which makes it easier to rank models and narrow a shortlist. That convenience is also the trap. A benchmark compresses every task into a single score, and real work refuses to be compressed.
Look closely at that collapse and you notice the failures change character, not just frequency. On synthetic benchmarks, models fail with logic errors — the kind of slip you would expect from a brilliant intern who misread the spec. On real-world tasks, they fail by misunderstanding context, missing implicit requirements, and making confident wrong assumptions about the broader codebase [1]. The same skill set produces a different species of mistake the moment the task changes shape.
And the shapes really are different. Synthetic tasks are short, clean, self-contained, reversible — a sealed puzzle with all the pieces in the box. Real software is long, multi-file, interrupt-driven, riddled with dependencies nobody bothered to write down. The benchmark stripped every one of those properties out and handed you a tidy number for a job that was never tidy.
Choosing a model by that number is a bit like hiring a chef based on a written exam about knives. Impressive score. No idea whether dinner arrives.
Work Has Geometry, and It Matters More Than Rankings
Once you stop asking which model is smartest, the structure underneath the work starts to surface. It turns out work has shape: short versus long, mixed versus focused, interrupt-driven versus sustained, reversible versus expensive to get wrong.
Some work is broad and mixed. It involves drafting, retrieval, tool use, structured output, dozens of small decisions, and constant switching between modes — the cognitive equivalent of working a busy service counter where the next request never resembles the last. Other work is deep rather than broad. It involves long documents, large codebases, multi-step research, extended reasoning chains, and sustained coherence held across hours of context — less a counter, more a long dive where surfacing too early ruins the whole attempt.
These are not the same job, and they do not reward the same architecture. Frontier models have largely reached parity on the raw scores, yet they diverge in philosophy: some optimize for centralized, long-context deep work, others for broad, versatile, rapid switching between tasks [7]. That divergence is not a flaw to be averaged away on a leaderboard. It is the entire point.
What strikes me is that the most fluent users already act on this without needing the theory. They reach for one system to run sustained research and a different one to move through development — not out of brand romance, but because the work has a shape and they have learned to feel it in their hands [10]. They stopped asking which model is smartest. They started asking what kind of cognitive architecture this particular workflow needs.
When Frontier Models Meet Real Work
The cleanest proof I have seen comes from TheAgentCompany, a 2025 benchmark that did the thing most evaluations carefully avoid: it made the models do the actual job. Not answer trivia, not generate snippets — file pull requests, update project trackers, analyze financial data, produce reports across 175 consequential enterprise tasks [4].
If intelligence were truly one-dimensional, performance should land roughly even across those domains. It did not, and the spread is the whole story.
The best model finished only 30% of the tasks. More revealing than the ceiling was the distribution: near-zero on finance and data science, meaningfully higher on software engineering [4]. The researchers’ own conclusion was blunt — the shape of the work, not the intelligence of the model, was the dominant predictor of whether it succeeded or face-planted.
This changes the order of the selection work. If task type is what predicts success, then your model shortlist matters less than your task map. Select by overall benchmark score and you will reliably pour your strongest model into the workflows whose shape it is worst suited for — then file the disappointing results under “adoption challenges” and schedule another training.
The Market Is Already Voting
Practitioners are arriving at task-specific selection through experience, even without a formal framework.
In Andreessen Horowitz’s 2025 survey of enterprise CIOs, 37% reported running five or more models simultaneously, up from 29% the year before [3]. The driver was not vendor risk or procurement hedging. It was use-case differentiation — different work needed different models, and they figured that out through operational experience rather than a strategy offsite.
The infrastructure is following the same logic. IDC projects that 70% of top AI enterprises will run dynamic model routing by 2028 [2], and the major providers already ship their products as mixtures of experts — routing your request to a specialized sub-model behind a single friendly interface [6]. You have been doing task-shape routing for a while now. They just never made you watch it happen.
The economics close the case. Intelligent routing — sending each query to the model that actually fits it — cuts costs by roughly 39%, trims usage 37-46%, and improves latency 32-38% [5]. The whole conversation across 2026 has shifted accordingly, away from raw model power and toward deployment, governance, and organizational design [9]. Matching model to shape is not just more effective. It is the cheaper option too, which is rare enough to be worth noticing.
Maybe the Gap Closes on Its Own
I want to give the case against all of this in its strongest form, because it deserves one.
Start with convergence. If frontier models keep climbing until one of them scores 95% on everything, then task-shape routing becomes pointless overhead — selection logic and a tangle of integrations maintained for a gap that no longer exists. This is a real bet, and it might pay off; capability curves have outrun skeptics before. But you cannot deploy on a forecast. The best model still tops out at 30% on real enterprise work and runs near-zero across whole domains [4]. So the move is to hedge the trajectory rather than bet against it: build routing that is lightweight and reversible, cheap to unwind the day a single model genuinely does everything. Done that way, the convergence bet costs almost nothing if it lands and saves a great deal if it stalls.
Then there is maturity. The objection that most organizations are simply too early for this is honest, and partly right. But it confuses the idea with its most elaborate implementation. Task-shape thinking begins with awareness, not a routing platform — just noticing that “quick mixed tasks” and “deep sustained work” are different animals that happen to share a cage. And with 37% of CIOs already running five or more models [3], this is mainstream practice, not aspirational architecture.
Finally, the benchmark defense, which I find the most respectable of the three. Benchmarks are standardized, reproducible, and cheap, and no rival framework matches that combination. Agreed — the argument was never to set them on fire. They are a necessary first filter and an insufficient final verdict. A fifty-to-sixty point gap between synthetic and real performance [1] is not a calibration quirk you can mentally adjust for over coffee. It is a precise measurement of how much shape the benchmark threw away. Use the leaderboard to narrow the field, then test on your real work before you commit a budget to it.
Matching Cognitive Architecture to Workflow Geometry
So what do you actually do with this? Here is a framework you can run without buying a single thing.
Classify. Map each major workflow along five dimensions: duration, mode, reversibility, attention pattern, and context dependency. They are independent — a task can be short but expensive to get wrong, or long but cheap to retry. Do this with the people who do the work, not the people who describe it from a deck. They feel the shape in a way no org chart captures.
Benchmark reality. Pull 10-20 representative tasks from actual production — not synthetic stand-ins — and test your candidate models on them. Score context handling and failure modes, not just accuracy. If a model dazzles on short-mixed work and disintegrates on long-sustained work, that is signal, not noise.
Route. Keep a portfolio of models for the different tasks. Match each task shape to the architecture that fits it, and start with manual routing — a human choosing at task initiation — long before you automate anything.
Instrument by segment. This is the step most teams skip, and it is the one that pays. A model averaging 70% might be running 95% on short tasks and 30% on deep ones. The aggregate hides the governance signal entirely. Measure fit by task shape, or you will keep mistaking a routing problem for a model problem and shopping for a new model you do not need.
Evolve. Models and work both shift shape over time. Re-run your reality benchmark quarterly so your routing stays matched instead of hardening into a habit no team remembers choosing.
The same logic surfaces in multi-agent systems, which become measurably more stable and accurate when each agent runs a model matched to its specific job — rather than one model gamely wearing five hats at once [8].
Where You Sit
The strongest AI users will not be the ones loyal to a single system. They will be the ones who understand that the right model depends less on brand and more on the geometry of the task.
So here is the move worth making this week. Take your three highest-volume AI workflows and classify each along the five dimensions, sitting with the people who actually run them. You will probably discover that work you have been treating as one job is really two or three different shapes wearing the same job title — and that the model topping your leaderboard is the right fit for, at best, one of them.
Stop asking which model is smartest. Start asking what shape the work is. A benchmark helps narrow the choice. The structure of the work tells you what to test next.
References
- [1] Rahman et al. (2025). Beyond Synthetic Benchmarks: Evaluating LLMs on Real-World Programming Tasks. arXiv:2510.26130.
- [2] IDC (2025). Why the Future of AI Lies in Model Routing.
- [3] Andreessen Horowitz (2025). Enterprise CIO Survey.
- [4] Xu et al. (2025). TheAgentCompany: Benchmarking LLM Agents on Consequential Real World Tasks. arXiv:2412.14161.
- [5] Requesty (2025). Intelligent LLM Routing in Enterprise AI: Uptime, Cost Efficiency, and Model Selection.
- [6] IDC (2025). Why the Future of AI Lies in Model Routing.
- [7] Neontri (2026). Claude vs GPT: Two Giants, Different Strengths.
- [8] JetBrains (2026). The Best AI Models for Coding: Accuracy, Integration, and Developer Fit.
- [9] Analytics Central (2026). AI Trends 2026: From Model Hype to Enterprise Reality.
- [10] Neontri (2026). Claude vs GPT: Two Giants, Different Strengths.