The pilot does not fail at the end. It fails in the first meeting.

“Understanding AI” series, note 8 of 12. Note 6 gave the order of the steps and note 7 put a price on each. This one explains what happens when they are not followed, with what has been measured by those who have studied hundreds of failed projects.

Thursday, 5:30 pm, a hotel group based in Interlaken. The board reviews the AI project it approved fourteen months ago. There was a pilot in one hotel, with an assistant that answered guests by chat. The report at the time spoke of 92% satisfaction. Since then a second pilot has been run in another hotel, the provider has been changed once and a person has been hired to “coordinate AI”. There is no hotel in which the system works in production. The chief executive proposes a third pilot, “this time with agents”. The board member in charge of finance asks how much has been spent. Nobody has the figure to hand.

The scene is imaginary. The pattern, the pilot that chains into another pilot, repeats in so many companies that it has studies devoted to it. In this note I give you the figures, the five causes those studies find again and again, the symptoms a board can see without understanding the technology and the way out. If you have read the previous notes, you will recognise each cause: it is the step that was skipped.

A pilot with no production date is not a pilot. It is an expensive demonstration renewed every year.

The figures.

In August 2024 the RAND Corporation published a report based on 65 interviews with data scientists and engineers with at least five years of experience in AI projects. Its starting point was an estimate it records with caution: by some estimates, more than 80% of AI projects fail, twice the rate of technology projects that do not use AI. A year later, in August 2025, the report by MIT’s NANDA project that I cited in note 6 measured it from another angle: 95% of generative AI pilots had produced no measurable impact on the profit and loss account. And Gartner, in June 2025, predicted that more than 40% of agent projects would be cancelled before the end of 2027.

They are three different measures of the same thing, and they should be read with care: “failing” means something different in each, and none is a controlled experiment. But the conclusion they share leaves no room for doubt. The normal outcome, in 2026, is for an AI project not to reach production. If yours does, it will not be because of the technology. It will be because five things were avoided.

Cause 1. Nobody understood the problem.

It is the first RAND finds and the one I see most. The project starts with the technology, “let’s do something with AI”, and looks afterwards for a problem to apply it to. Or it starts with a problem badly understood: management wants to reduce calls to support, the technical team builds an assistant to answer them faster, and nobody noticed that the calls existed because the invoice was confusing. An assistant that explains a confusing invoice very well does not reduce the calls. A clear invoice does, which is ordinary software, or paper.

The symptom, from the top: nobody can say, in one sentence, which number has to change and by how much. If the answer to “what do we measure?” is “satisfaction” or “efficiency”, the problem is not understood. In IBM’s May 2025 survey of 2,000 chief executives, 64% admitted having invested in technology before properly understanding its value, for fear of falling behind. That is cause 1 seen from above.

Cause 2. There was no data, or it was bad.

RAND’s second cause is the lack of the data needed to train or feed the system. In today’s generative AI there is almost never anything to train, but the model does have to be given the right documents, and here the Harvard Business Review’s 3% from note 6 returns: most companies do not have their data in order. The assistant at the hotel in Interlaken answered with the 2022 cancellation policy because that was the one somebody uploaded. When a guest complained, the team blamed the model.

The symptom: the provider asks for the documents and the company takes weeks to gather them, or hands over seven versions. Step 2 from note 6 was not done, and no model does it for you.

Cause 3. The technology was built and not the infrastructure.

RAND describes it as focusing on the latest technology instead of on what surrounds the model: the connection to the company’s systems, logging, version management, maintenance. It is item 6 from note 7, the one nobody budgets for. The pilot worked on the provider’s laptop; production required it to talk to the booking system, to keep a trace of every answer and to have someone maintain it. That was another project, and nobody had approved it.

The symptom: the demonstration is impressive and the question “how does it connect to our booking system?” gets a “that comes in the next phase”. The next phase is the real project. The pilot was the easy part.

Cause 4. The problem was too hard for today’s AI.

RAND’s fourth cause is the most uncomfortable: there are problems today’s AI does not solve, and organisations choose them because they are the most attractive. You will remember the jagged frontier from note 4 and the one-hour horizon from note 5. An assistant that answers a hotel’s frequently asked questions is inside the frontier. One that negotiates compensation with a guest for a noisy room, applying judgement, policy and tone, is outside it. The Interlaken pilot measured satisfaction on the first kind of conversation and failed in silence on the second, where guests ended up calling reception.

The symptom: the task chosen is the one that excites people most, not the one that repeats most. And the 92% satisfaction does not say which conversations it was measured on.

Cause 5. The organisation did not change.

The fifth cause, which RAND states as the lack of an infrastructure of people and processes, is the one that explains most of the pilots that technically worked. The system is fine; nobody uses it, or it is used without trust, or it is used and nobody reviews. It is the 50% adoption from the sum in note 7, the one that turned six months of payback into thirteen. The Klarna case I told in note 4 is the most visible version: the technology did its part, and the company discovered the following year that it had removed people from where they were needed.

The symptom: the project has a technical owner and no business owner with the number in their annual objectives. When the pilot ends, the technical owner moves on to the next pilot, and the process goes back to being done as before.

The eternal pilot trap.

There is a sixth cause RAND does not list because it is not a technical cause, and which I put first: the pilot had no production date. A pilot is an experiment with a question, a deadline and a decision at the end: it goes into production, it is changed or it is closed. A pilot without those three things is a comfortable place where an organisation can say it is “doing AI” without risking anything, and where a provider can invoice phases. Fourteen months and three pilots are the complete symptom.

The way out is mechanical and is in note 6: a process with a number that is bad today, fifty gold cases, shadow mode with an end date, tiered review and a business owner. Shadow mode is the pilot done properly: it has a question (“how many out of every hundred does it get right?”), it has a deadline (“four weeks”) and it has a decision (“if it gets more than 90% right on the safe cases and sends the doubtful ones to people, it is switched on on the 1st of the month”). It is no more expensive than a pilot. It is a pilot that ends.

• A number before a technology. If nobody can say which figure changes and by how much, it is not approved.
• Documents before demonstration. Until the archive is clean, any demonstration measures the provider, not your company.
• Production from day one. The connection to your systems, logging and maintenance are budgeted in the proposal, not “in the next phase”.
• Inside the frontier. The most repeated and boring process, not the showiest. The showy one comes later, with what was learnt.
• An owner with the number in their objectives. From the business, not technical. With the authority to change how their team works.
• Date and decision. Every pilot ends in one of three words: switch on, change or close. And the date is written before it starts.

1. Today

   Ask for the figure nobody had to hand

   How much has been spent on AI in the last two years, including licences, consulting and internal hours. Compare it with what is in production.

2. This week

   Do the one-sentence test

   For every project under way: which number changes, by how much and by when. Whatever does not fit in a sentence stops until it does.

3. This month

   Turn every pilot into a shadow mode with a date

   Four weeks, fifty cases, a hit-rate threshold and one of three decisions at the end.

4. This month

   Name a business owner per process

   With the number in their annual objectives. The technical owner helps; they do not decide.

How I approach it.

I do not run pilots. I run four-week shadow modes, with the number written beforehand and the decision fixed at the end, and the project includes from day one the connection, the logging and the maintenance, because without them there is no production, there is a demonstration. When a company shows me its third pilot, the first thing I ask for is the spending figure. Almost always, with that figure, the first production process would already have been paid for twice.

Think of your business’s last pilot, in AI or anything else. Did anyone write down, before it started, which number had to come out for it to continue? And does anyone remember what it was?

If the answer is no, perhaps the pilot did not fail. Perhaps it never had a way to succeed. The unsettling part is that this was decided in the first meeting, and almost nobody remembers deciding it.

Next note: the law. What the European AI Act requires of your company, what changes in Switzerland, what data protection says about pasting a report into an assistant and the dates that really affect you, without alarm and without legal advice.