Why most AI pilots die, and what the surviving 10% did differently

We have now run or reviewed enough AI engagements across Thailand and the region to see the same story repeat. A pilot is commissioned with genuine enthusiasm, demos well in month two, and is quietly switched off by month nine.

The model is almost never the reason.

#Failure pattern one: choosing the impressive workflow

The workflow that demos best is rarely the workflow worth automating. Executive dashboards, natural-language querying of the data warehouse, and "ask the company anything" assistants all present beautifully in a steering committee and deliver very little measurable time back.

The workflows worth doing are boring:

• The morning reconciliation between three systems that occupies four people for ninety minutes
• The product copy that gets rewritten from scratch for every marketplace at every launch
• The first-response queue that takes eleven hours when the shift is thin

None of these are exciting. All of them have a number attached, and that number is what survives a budget review.

We now open every engagement by timing tasks and ranking them on hours saved per baht spent. The client keeps that ranking whether or not they hire us, because it is genuinely the most useful artefact in the whole process.

#Failure pattern two: no evaluation harness

A prompt is code that fails silently. Change a sentence, and a whole category of answers can degrade without a single error being thrown.

The fix is unglamorous and cheap: a golden set of representative inputs with known-good outputs, run automatically on every change.

eval: product-listing-th
  cases: 84
  pass:  81   (96.4%)
  fail:  3
    - "sunscreen SPF50 refill" → dropped size variant
    - "gift set 3-piece"       → wrong category mapping
    - "limited edition"        → tone breach: informal

Three failures out of eighty-four is a report you can act on in an afternoon. The same three failures discovered through customer complaints is a week of firefighting and a loss of internal trust that takes months to rebuild.

If a vendor cannot show you their eval suite, they do not have one.

#Failure pattern three: shipping a new destination

Every new interface is a new habit, and habits are the expensive part of any rollout. An agent that lives at its own URL requires a person to remember it exists, log in, and choose it over the thing they already do.

Systems that survive are the ones that appear where work already happens:

• Inside LINE, because in Thailand that is where the conversation already is
• Inside Google Sheets, because the ops team never left it
• Inside the CRM record the salesperson already has open
• As a scheduled job that simply produces the output, with no interface at all

The best agent interface is frequently no interface. If the reconciliation is done before anyone arrives and only the exceptions are flagged, nobody needs to be trained on anything.

#Failure pattern four: permanent dependency

A system whose prompts can only be changed by the agency that built it will be switched off the first time the agency's retainer comes up for review. This is not cynicism about clients — it is a straightforward observation about what survives a budget cycle.

Our handover is deliberately complete:

1. The repository, with the deployment scripts and runbook
2. The prompt library, versioned, with the rationale for each constraint
3. The evaluation suite and the last full report
4. Two named people inside the business trained to run all of the above
5. The specification for the next workflow, so the internal team has an obvious second project

If you still need us in month twelve, we built it wrong.

That position occasionally costs us follow-on revenue. It has also meant that the systems we ship are still running, which turns out to be a better business development strategy than lock-in.

#The shape of an engagement that works

Ninety days, one workflow, fully handed over. Then the client decides whether the second one is worth doing — and by that point they have real evidence rather than a vendor's projection.

#What this means if you are commissioning AI work

Ask three questions of any partner:

1. What is the hours-saved number, and how did you measure it? If the answer is qualitative, the project has no defence at budget time.
2. Can I see your evaluation suite? If it does not exist, reliability is a hope rather than a practice.
3. What does handover include? If the answer does not include the prompts, you are renting, not buying.

Any partner who answers all three clearly is worth talking to further. Most cannot answer the second one.