Organisations run AI pilots that succeed and then do not deploy them. The pattern is common enough to be predictable, and the failure sits between the pilot and production rather than inside either.
What a pilot is designed to prove
A pilot answers whether a model can perform a task acceptably, under favourable conditions, on representative examples.
That is a real question and worth answering, since a substantial share of proposed applications fail it.
The pilot is deliberately unrepresentative in other respects. It runs on a curated dataset, is operated by people who understand it, and has an engaged sponsor watching.
None of those conditions holds in production, and none of them was supposed to.
The mistake is treating a passed feasibility test as evidence that deployment will be straightforward.
Why the pilot dataset is clean
Pilot data is assembled by hand, and assembly involves selection.
Malformed records, ambiguous cases and examples nobody could label confidently get dropped, usually without a decision being recorded.
The retained set is therefore easier than the live stream, sometimes considerably so, and accuracy measured on it is optimistic.
Live data also drifts. Formats change, upstream systems are modified, and new categories appear that the pilot never saw.
Teams that sample the production stream at random for their pilot get worse numbers and far fewer surprises later.
The ownership gap after the pilot
A pilot is run by an innovation team, a consultancy or a small group of enthusiasts. Production systems belong to operational teams with existing responsibilities.
The handover is where projects die, because the receiving team has no capacity allocated, no expertise in the system, and no involvement in its design.
They also carry the risk if it fails, which makes reluctance rational rather than obstructive.
Projects that survive typically had the operational owner participating from the start, with time formally allocated to it.
Ownership is an organisational question and is usually treated as a technical one, which is why it is discovered late.
Why integration is the actual project
A pilot reads a file and writes a result. Production reads from systems of record, writes back to them, handles failures, logs decisions and respects access controls.
That work is ordinary software engineering, it is substantial, and it is invisible in a demonstration.
Legacy systems complicate it further, since many have no interfaces designed for programmatic access and were built when today's requirements did not exist.
Integration effort routinely exceeds the model work by a wide margin, and budgets are usually written the other way around.
The gap between those two numbers is the most common reason a funded pilot becomes an unfunded deployment.
How evaluation gets deferred
Pilots are evaluated by people looking at outputs and forming a judgement, which works at small scale.
Production needs continuous measurement, because quality degrades quietly as inputs shift and nobody is watching every output.
Building that measurement requires deciding what correct means precisely enough to check automatically, which is genuinely hard and frequently postponed.
Without it, the system runs unmonitored until a visible failure prompts an investigation, and by then trust is gone.
Teams that define their evaluation method during the pilot deploy more often, because they can demonstrate control rather than asserting it.
Why the business case moves
A pilot business case estimates savings from time no longer spent on a task.
Realising that saving requires the freed time to be redeployed, which means changing roles, workloads or headcount, and those changes need approval from people who were not in the pilot.
Absent that change, the work simply gets done with less effort and the saving never appears in any account.
New costs also arrive: licences, monitoring, review of outputs, and the staff time to handle cases the system escalates.
A case built on gross time saved rather than net cost change tends to collapse when finance examines it, and finance examines it at the deployment gate.
What a pilot that deploys looks like
Deployable pilots are narrow. One task, one team, one measurable outcome, rather than a platform intended to serve many uses.
They run on live data from the start, accepting worse initial results in exchange for honest ones.
They include the operational owner, a named evaluation method and a rough integration estimate before the first model call is made.
They also define a failure condition in advance, which makes stopping a decision rather than a defeat.
These constraints make the pilot less impressive and considerably more likely to become a system.
Where the pilot should have started
The strongest predictor of deployment is choosing a task where the current process is already well understood and measured.
If nobody knows how long the task takes today, how often it is done wrong, or what happens downstream when it is, there is no baseline to improve against and no way to prove the system helped.
Processes in that condition need documenting before they need automating, and the documenting frequently reveals cheaper fixes than any model.
The instinct to pilot AI on the most visible problem is understandable and works against this, because visible problems are usually visible precisely because they are tangled.
Starting on a boring, measured, well-owned task produces a working system and an organisation that has learned how to build the next one, which is worth more than the first deployment itself.