Why an AI needs a map to navigate
Trusting an AI that acts on its own does not depend on the AI: it depends on the company having written down what it means by its own words.
Can you trust an AI that is going to act on its own?
In fact, that trust does not depend on the AI itself, but on writing down clearly what it means when it uses your own words.
Every company knows what an “active customer” is, knows what counts as “urgent”, knows when a case is “resolved”. It just knows it the way we know how to ride a bicycle, in the flow of work, and that is often not explicit. It is knowledge that lives in the heads of people who have worked there for years, in decisions made by common sense, in exceptions everyone understands without needing an explanation, and it never had to be written down because there was always a human nearby to interpret the context.
That stopped being enough the moment we had at our disposal a technology that went from merely answering to acting.
While it only suggested an answer and someone checked before sending, an error was annoying but reversible. Now it can approve the refund, cancel the contract, prioritize the ticket, fire off the message. By the time someone notices, the error has already happened, and the review interval we always counted on as a safety net is gone.
A new employee stops and thinks when faced with a question: “does this case count as urgent or not?” That pause costs a few minutes, but it is essential to avoid the mistake.
The AI does not stop, and it does not know that it does not know. It produces an answer with the same confidence as always, whether it is right or completely wrong, because there is no signal in it saying “I am not sure about this one, can someone check?”. It decides and moves on.
And the problem is not that it errs once; we can all err too. The problem is that it can err systematically, always in the same direction, at machine speed, without setting off any alarm.
If the definition of “important customer” it learned is a little off, it will not mistreat one customer once; it will mistreat every customer who resembles that one, every day, in silence, until someone notices a strange pattern in the numbers or, worse, until someone feels it firsthand.
That is why defining what a word means inside your company should no longer be just another technical task, but a fundamental decision.
Someone decided, at some point, what to do with the customer who pays late but has been there for ten years. Someone decided what counts as a serious complaint and what is just a customer's bad day. Those decisions carry judgment, carry context, carry the experience of whoever answers for the consequences.
When you hand operations over to an AI without ever having formalized those decisions, you are outsourcing a judgment nobody ever made on purpose to a system that will make that judgment one way or another, with or without your participation.
Do you know what your company means by the words it uses every single day? Is that written down somewhere, or are you hoping the AI will understand it the way you would?
The rush to put AI into action is running ahead of the most basic question, the one that should come first. Before asking what the AI will do on its own, it may be worth asking: what have we never stopped to write down?