The gym before the marathon.
Every few weeks a new report lands on the same conclusion: most enterprise AI pilots never make it to production. The latest, from MIT, put a number on it — roughly 95% of corporate generative-AI pilots produced no measurable return on the P&L [1]. RAND found that more than 80% of AI projects fail, about twice the rate of regular IT projects [2]. The boardroom takeaway writes itself: be careful, most of this doesn’t work.
It’s true. And it’s the wrong lesson.
Because “most pilots don’t reach production” is the same shape of sentence as “most people who join a gym never become Olympic athletes.” Also true. Also not an argument against going to the gym.
The conclusion doesn’t follow from the premise. The purpose of early training was never immediate performance. It’s adaptation.
Most AI advice assumes a skill the organization doesn’t have yet
Read enough strategy decks and you notice a quiet assumption buried in all of them: that the organization already knows how to work with AI, and the only question left is where to point it.
Learning to work with AI is part of the transformation; the strategy cannot simply assume that capability is already in place.
Knowing how to work with AI is not a slide. It’s a motor skill — learned the way every motor skill is learned, by doing the thing badly and then less badly. You can’t read your way into it any more than you can read your way into a faster mile.
So when a consultant says “we shouldn’t run pilots until we have a proven path to production,” picture the same advice at the gym: we shouldn’t start running until we have a proven path to a marathon. The gym comparison shows what that prerequisite would prevent. You run because running is how the body learns to run. The lungs adapt. The legs adapt. The technique sharpens. Your sense of your own limits gets accurate. Only after all that does the word “marathon” mean anything.
AI transformation works the same way. The first value of a pilot is not the pilot’s outcome. The first value is the adaptation it forces.
What pilots actually produce (and it isn’t ROI)
Ask an executive what a pilot is supposed to produce and you’ll hear: ROI, an automation, a business case. Sometimes you get them. But the most valuable output is usually the one nobody put on the scorecard.
Human muscle memory. The people in the pilot learn how to prompt, how to review output they can’t fully trust, where the model quietly fails, where it surprises them, and how to divide work between a person and a system that is confident and wrong in equal measure. None of that transfers from a PowerPoint. It only comes from reps.
Organizational muscle memory. The company learns where its data actually lives (rarely where the architecture diagram claims). Which workflows matter and which just feel important. Where an approval silently breaks. What governance a real deployment demands rather than the version the policy doc imagined. Which capabilities turn out to be reusable.
This is exactly the gap the numbers describe. BCG found only about a quarter of companies have moved beyond proofs of concept to real value [3]; McKinsey’s surveys keep showing near-universal adoption alongside scarce enterprise-level impact [5]. The constraint isn’t the technology. It’s an organization that hasn’t trained yet — and you can’t shortcut training by buying a better treadmill.
The problem was never pilots. It’s isolated pilots.
Here’s the distinction that the failure statistics miss entirely.
A pilot that ends as a one-off experiment is wasted effort. A pilot that leaves something reusable behind is organizational learning. Same activity. Opposite value. And the difference is invisible to any metric that only asks, “Did it scale?”
That’s the question most transformation programs ask, and it’s why Gartner expects at least 30% of generative-AI projects to be abandoned after the proof-of-concept stage [4]. The pilots get graded like products — ship or die — when they should be graded like training sessions: what did we get stronger at?
Consider two companies running the same twelve pilots. (An illustrative composite, not a single firm.)
The first treats each as a standalone bet. Eleven don’t scale. The eleventh review meeting concludes, reasonably, that “AI isn’t ready for our business.” They were right about the pilots and wrong about everything else.
The second treats each pilot as a deposit. Every experiment leaves behind its prompts, its evaluation sets, its reworked workflow, its data plumbing, its governance pattern. By the fifth pilot, teams start from a library instead of a blank page. By the twelfth, the thing ships in weeks — not because the model got better, but because the organization did. Same hit rate on individual pilots. Completely different trajectory.
The mechanism is compounding. The first company ran twelve experiments. The second ran one experiment twelve times.
Here is the catch
The honest objection: pilots aren’t free. Run a dozen scattered experiments with no discipline and you’ll burn budget, exhaust goodwill, and train the organization to roll its eyes at the next AI initiative. “Run more pilots” can be an excuse to avoid the hard work of committing to one and making it real. Discipline, the argument goes, means killing pilots faster — not starting more.
That’s right about the failure mode and wrong about the cause. The cost to watch is the pilot that leaves nothing behind. A pilot still uses time and budget; reusable assets change what the next experiment needs to spend. A learning-system pilot is cheap precisely because the next one inherits its assets — the expensive part was already paid once. The discipline isn’t “run fewer.” It’s “run none in isolation.” Pick experiments in workflows that matter, and require each one to deposit something the next can stand on. Kill the experiment, keep the capability.
A test you can apply on Monday: what did the pilot leave behind?
Stop scoring pilots on whether they reached production. Score them on what they deposited into the organization’s permanent capability. Every pilot, win or lose, should leave behind:
- Prompts that worked, and the ones that didn’t, with notes on why.
- Evaluations — the test sets and rubrics that tell you whether output is good, reusable on the next project.
- Workflows — the redesigned process, not just the model call.
- Data structures — what you had to clean, connect, or expose.
- Agent patterns — the orchestration that held up under real conditions.
- Governance patterns — the approvals, guardrails, and audit trails a real deployment needed.
- Lessons — written down, where the next team will actually find them.
If a finished pilot leaves none of these, it failed — even if it shipped. If it leaves most of them, it succeeded — even if it was killed. Run the audit on your last three pilots. The pattern will tell you whether you’ve been training or just spending.
What survives the pilot
So what actually remains after the experiment is over?
Not the model — that gets swapped within a year. Not the chatbot. Not the proof of concept. What survives is the organizational learning: the prompts, the workflows, the evaluation systems, the judgment, the reusable agent patterns, the accumulated know-how.
That residue has a name. It’s token capital — the compounding asset a company builds as it learns to turn intent into working systems. The pilot is not the product. The pilot is the machine that converts human capital into token capital, one repetition at a time.
This is why the iteration matters more than the automation. A business’s biggest AI advantage isn’t any single thing it automates — it’s the rate at which it can try, learn, and try again. Pilots are how you actually pay for that rate. Each one is a rep. The capital accrues in the reps, not the wins.
Most executives are worried their AI pilots aren’t scaling. I’d worry more about the ones not running enough of them. The goal of a pilot was never to prove that AI works. The goal is to teach the organization how to work with AI.
An athlete doesn’t get stronger by writing a training plan. A company doesn’t become AI-native by writing one either.
References
- [1] MIT (Project NANDA / Media Lab), The GenAI Divide: State of AI in Business 2025 (Aug 2025) — ~95% of enterprise generative-AI pilots delivered no measurable P&L impact.
- [2] RAND Corporation, The Root Causes of Failure for Artificial Intelligence Projects (2024) — >80% of AI projects fail, roughly twice the rate of non-AI IT projects.
- [3] Boston Consulting Group, Where’s the Value in AI? (Oct 2024) — only ~26% of companies have moved beyond proofs of concept to tangible value.
- [4] Gartner, press release (July 2024) — at least 30% of generative-AI projects will be abandoned after proof of concept by the end of 2025.
- [5] McKinsey & Company, The State of AI (2024–2025) — near-universal AI adoption alongside scarce enterprise-level EBIT impact; value concentrates where workflows are redesigned.