The Great Human Hunt: A 2025 Customer Service Story

How many times have you shouted this at a chatbot?
I’ve done it more times than I want to admit.

In my work, I also get to switch sides and look at the teams providing these systems, or sit with the engineering team behind them.

Usually, to them, everything looks fine. The AI performance metrics look good. The dashboards are clean. Everyone feels quietly confident that things are “good enough.”

But the moment you look at actual outcomes –
real customer satisfaction,
real escalations,
real decision quality,
you realise something is clearly not working the way people assume it is.

And honestly, after seeing this across so many companies, the pattern is impossible to ignore.

The model is almost never the real problem.

I keep running into the same three issues again and again:

𝟏. 𝐃𝐚𝐭𝐚 𝐈𝐧𝐭𝐞𝐠𝐫𝐢𝐭𝐲
Teams argue about definitions that should be obvious, and the model ends up learning from contradictory truths.

𝟐. 𝐃𝐞𝐜𝐢𝐬𝐢𝐨𝐧 𝐂𝐥𝐚𝐫𝐢𝐭𝐲
Ask three people how a decision is made today and you’ll get five answers.
AI learns those contradictory, unwritten rules… inconsistently.

𝟑. 𝐄𝐯𝐚𝐥𝐮𝐚𝐭𝐢𝐨𝐧 𝐀𝐫𝐜𝐡𝐢𝐭𝐞𝐜𝐭𝐮𝐫𝐞
Everybody checks the model before launch. Nobody checks it after. So drift quietly creeps in until customers are the first to notice.

𝐓𝐡𝐢𝐬 𝐢𝐬 𝐭𝐡𝐞 𝐡𝐢𝐝𝐝𝐞𝐧 𝐜𝐨𝐬𝐭 𝐨𝐟 “𝐠𝐨𝐨𝐝 𝐞𝐧𝐨𝐮𝐠𝐡” 𝐀𝐈.

It behaves well in the metrics… and badly in the real world.

I wrote about this in my latest Substack because these problems are fixable, but only if you stop looking at your dashboards and start examining your foundations.

If you’ve ever felt like your AI is “mostly fine” but your customers are telling a different story… you’ll relate to it.