A Chipotle customer recently asked the company’s ordering chatbot to reverse a linked list in Python. The chatbot obliged — wrote clean code, explained the logic step by step, then politely circled back to ask what the customer wanted for lunch.
A fast-food ordering bot, built to sell you a burrito, just delivered a computer science tutorial before getting to toppings.
This is not a bug. It is the inevitable result of plugging a general-purpose language model into a single-purpose system. The model can do anything, so it does anything — including things the customer never asked for and nothing in the business case requires. And it points to something deeper than a funny screenshot: most companies deploying AI right now are making the same mistake at much higher stakes.
I keep hearing the same question from product leaders — in board meetings, on roadmap calls, sometimes at conferences where the slide deck is already approved. “Why would anyone use ChatGPT when every company will have its own AI?” It is the wrong question. The right question: what kind of intelligence are you embedding, and where does it actually belong?
The Architecture Question Behind the Chatbot
Two things are being conflated across the industry, and the confusion is expensive.
General-purpose models — ChatGPT, Claude, Gemini — are broad reasoning engines. They handle ambiguity, interpret novel input, solve cross-domain problems. Task-specific systems — ordering flows, support desks, compliance workflows — are narrow by design. They need structured inputs, deterministic outputs, and clear boundaries. They are good at something different entirely.
The industry default is “put a chatbot in front of this.” It works in demos. In production, it creates systems that can answer anything but probably should not.
The failure rate is the tell. Seventy-four percent of enterprise CX AI programs failed to generate measurable value by late 2024, and only one in four made it past the pilot stage. Gartner predicted 30% of generative AI projects would be abandoned after proof of concept by end of 2025, with over 40% of agentic projects canceled by 2027. A Berkeley analysis in early 2026 sharpened it: only about 10-15% of companies achieve measurable business impact, and nearly 85% of initiatives never reach full production. This is a pattern with a shape — and the shape points to architecture, not technology.
Natural language interfaces are genuinely valuable. The problem is that companies use the language model as the entire system: comprehension, reasoning, policy application, and response generation collapsed into a single undifferentiated layer. That is not a natural language interface. That is an architecture decision no team chose on purpose.
The Chipotle Problem Is Everywhere
I call it the Chipotle Problem: taking general-purpose intelligence and dropping it into a narrow workflow without redesigning the system around it. The burrito bot is the funny version — overpowered enough to explain Python, yet it still needs a button for “Where is my order?” The expensive versions are running in production right now.
AI-powered customer service fails at roughly four times the rate of other AI applications, and nearly one in five consumers report zero benefit from it. That is not a teething problem. That is a design failure at scale.
Consider Klarna. The Swedish fintech replaced approximately 700 customer service employees with an AI chatbot that handled 2.3 million conversations per month. The company announced $10 million in savings and became the poster child for AI-driven workforce transformation.
Then customer satisfaction dropped. Users reported robotic responses, inflexible handling, Kafkaesque loops of repeating their issue to a human after the bot failed. Klarna still uses AI for straightforward queries — billing lookups, basic account questions, the kind of thing a flowchart could handle. But by mid-2025, CEO Sebastian Siemiatkowski publicly reversed course: “We focused too much on efficiency and cost. The result was lower quality, and that is not sustainable.”
The lesson is not that Klarna failed. It is what the AI could and could not do. It could talk fluently. It could not judge when a customer’s frustration required a human, when a policy exception was warranted, or when rigid script-following was making things worse. Language fluency did not supply the judgment those support situations required.
Air Canada escalated the same failure from bad UX to legal liability. The airline’s chatbot told a passenger he could apply for a bereavement fare discount retroactively within 90 days. That policy did not exist — the chatbot invented it, with the confidence of someone reading from a manual. The British Columbia Civil Resolution Tribunal held the airline liable, ruling that “a company is responsible for all information on its website, whether it came from a static page or a chatbot.”
The chatbot understood the question perfectly. It just made up the answer.
Companies iterate — that is genuinely how adoption has worked, from databases to mobile to cloud. But iterating with AI in production is different. Hallucinated policies create legal liability; replaced agents create real customer harm; and the feedback loops that catch errors in traditional software — regression tests, code review, staging — do not exist for a system that generates novel output on every interaction. One estimate put H1 2025 AI investment at $47 billion, with 89% of that spend delivering minimal returns. That is not normal iteration cost. It is avoidable damage from embedding intelligence without designing the system it lives in.
Match Intelligence to the Task
The objection that almost works: general models handle edge cases that rigid systems cannot. A rule-based ordering bot handles “I want a burrito with no beans” but breaks on “I have a severe allergy to cilantro — can you tell me which items are safe?” General-purpose intelligence handles the long tail of human communication precisely because it is unconstrained. The Chipotle chatbot writing Python is embarrassing, but it also means the system gracefully handles literally any input a customer throws at it.
Handling ambiguous and unusual requests is a real advantage.
But it conflates two distinct capabilities: understanding what a customer means, and giving correct answers about what the business offers — parsing versus reasoning about business reality. Air Canada’s chatbot parsed the bereavement question perfectly; it failed by inventing a policy. The risk is not that general models cannot handle edge cases. It is that they handle them too confidently, generating plausible answers with no grounding in what is actually true.
This is where the Intelligence Placement Matrix matters. At every decision point in your system, ask one question: is this a judgment call or a transaction?
Judgment calls — ambiguous input, novel situations, interpretation required — belong with general-purpose intelligence. Transactions — deterministic, bounded, auditable — belong with structured systems. The same interaction often starts as a transaction and shifts into a judgment call mid-conversation: someone checking their order status gets frustrated about a late delivery, and now you need empathy, flexibility, maybe a policy exception. That fluid boundary is why seam design matters more than the model you choose.
The discipline is not “where can we add AI?” It is “where is AI the only thing that works?” Use the least powerful intelligence that solves the actual problem. A regex that extracts dates from a known format beats a language model for that task — every time, at a fraction of the cost. A lookup table that maps product codes to categories beats a classifier if your codes are stable. This is the same engineering principle that has governed system design for decades: use the right tool for the layer you are working in. Reserve general-purpose intelligence for genuinely open-ended problems where nothing simpler will do.
The accuracy gap backs this up. In one enterprise benchmark, custom-built specialized agents were roughly 20% more accurate and 40% faster than general-purpose ones — around 80% versus 59%. Deterministic and agentic approaches sit on a spectrum, and effective implementations match the approach to the problem rather than to whatever the industry is excited about this quarter. The Edge AI and Vision Alliance put it more provocatively in 2025: “the most impactful AI systems of the future will not be the largest, but the smallest” — purpose-built models that support real-time inference on constrained hardware and integrate with existing automation pipelines.
Then design the seams explicitly. Interpretation seams where unstructured input becomes structured output. Escalation seams where deterministic workflows hand off to model reasoning. Validation seams where model output gets checked against the system of record before anyone sees it.
Even if models become perfectly reliable tomorrow, the architectural question remains. Should all your business logic route through an opaque neural network? Better models reduce hallucination risk but do nothing for auditability, compliance, or cost. And “more capable” does not automatically mean “more controllable”: a 2025 Nature study found that fine-tuning GPT-4o on a single narrow task — writing insecure code — induced broad misalignment in unrelated contexts, including violent advice on benign questions. Specializing a general model does not automatically make it safer. The argument is about architecture, not capability.
You do not need Python in your burrito. You need the right ingredient for each layer.
What the Companies Getting It Right Do
Morgan Stanley asked a different question. Instead of “where should we put AI in front of our clients?” they asked “where do our people waste time on low-judgment work?”
The answer was specific: financial advisors spent enormous hours searching hundreds of thousands of internal research documents, writing post-meeting notes, and drafting client emails. So Morgan Stanley built three targeted AI tools — all serving the advisor, not the client. An internal document retrieval system that, per the firm’s reporting, increased search efficiency from 20% to 80%. A meeting debrief tool that auto-generates notes and action items. A research synthesizer for complex multi-product analysis.
Over 98% of advisor teams use the tools voluntarily, not by mandate. Mandated rollouts hit high usage numbers; 98% voluntary means the advisors actually found it useful — which means the intelligence was placed where it reduced friction rather than where it looked impressive on an earnings call. The contrast with Klarna is the argument in miniature: intelligence placed where expertise meets friction, not where the company meets the customer. AI as an internal force multiplier, not a client-facing replacement.
The pattern shows up elsewhere when you look for it. MIT Sloan Management Review documented CarMax using narrow LLM workflows to summarize customer reviews in hours instead of the weeks it previously required from multiple workers. BMW, profiled by Google Cloud, deployed small language models inside vehicles specifically because limited on-board hardware ruled out a foundation-model deployment. Different industries, same instinct: pick the smallest intelligence that fits the actual constraint. Without that redesign, AI amplifies existing flaws instead of solving them.
The Real Design Question
Every failed AI deployment I have seen shares the same root cause. Not bad models. Not insufficient data. Not inadequate training. A system design failure: intelligence placed in the wrong layer of the stack.
The question that separates strategy from theater is not “Do we have AI?” It is “Where should intelligence live in this system?” Three things to do before your next deployment:
Map every AI touchpoint and ask: is this general intelligence solving a structured problem, or structured logic handling a reasoning task? A language model deciding which database index to use is the Chipotle Problem. A flowchart trying to handle a customer who is upset and confused is the Chipotle Problem too — just inverted.
Redesign before you embed. Stop bolting AI onto unreformed systems. If the workflow was broken before AI, AI will break it faster and more expensively. The common thread in those $47-billion-worth of disappointing returns was not the technology — it was the absence of design work before the technology arrived.
Match intelligence to task shape. Route reasoning and edge cases to general models. Route transactions and state to deterministic systems. Design explicit handoff contracts at every boundary. The 80%-versus-59% specialized-vs-general gap is the difference between a system that works and one that demos well.
Right now a lot of systems are answering questions nobody asked — before getting around to what you want for lunch. The burrito bot did not need to know Python. Your system probably does not need general-purpose intelligence everywhere you have put it. The question was never whether to use AI. It was always where intelligence belongs — and answering it requires redesigning the system, not decorating it.
Sources
- Qualtrics XM Institute — AI customer service fails at 4x the rate of other AI applications; 1 in 5 consumers report no benefit
- Marily Nika / Substack — Chipotle chatbot wrote Python code to reverse a linked list, then returned to taking a burrito order
- Fast Company — Klarna replaced ~700 agents with AI (2.3M conversations/month), then reversed course after satisfaction dropped
- McCarthy Tetrault — Air Canada held liable for chatbot-hallucinated bereavement fare policy
- Eglobalis — 74% of enterprise CX AI programs failed to generate measurable value by late 2024; AI without system redesign amplifies existing flaws
- Gartner — 30% of GenAI projects abandoned after PoC by end 2025; 40%+ agentic AI canceled by 2027
- Berkeley Center for Market Research — Only ~10-15% of companies achieve measurable AI business impact; nearly 85% of initiatives never reach full production
- Alation — Specialized AI agents ~20% more accurate, ~40% faster than general-purpose (~80% vs ~59%)
- deepset — Deterministic and agentic approaches exist on a spectrum; match the approach to the problem
- Edge AI and Vision Alliance — “The most impactful AI systems of the future will not be the largest, but the smallest”
- Nature (2025) — Fine-tuning GPT-4o on narrow insecure-code task induced broad misalignment in unrelated contexts
- MIT Sloan Management Review — CarMax used narrow LLM workflows to summarize reviews in hours instead of weeks
- Google Cloud — BMW deployed small language models in vehicles with limited on-board hardware
- CMSWire — Industry estimate: $47B invested in AI in H1 2025; 89% delivered minimal returns