Why most AI chatbots don't improve your customer service (and what does)

You've probably been stuck with a chatbot that didn't help. You ask a specific question, get a smooth answer that misses the point, and keep clicking until you finally reach a person. That experience is why many companies see their customers get less happy after they launch a chatbot. The technology has improved a lot in the past few years. The experience on the other side of the conversation often hasn't kept up.

Why a chatbot often makes your customer service worse

Most AI chatbots are built to sound as helpful as possible. So they nearly always give an answer, even when they don't know it. A system that can't say "I don't know" fills the gap with something that sounds plausible. That's called a hallucination, and in customer service it's dangerous. Your customer relies on an answer with nothing behind it, and only finds out much later that it was wrong.

Your product, your prices and your terms change all the time. The knowledge a chatbot answers from changes much more slowly. A bot that takes its answers from an old export or a dated training run keeps giving last year's answer, with full confidence. The customer hears a sure answer that's simply out of date, and has no way to tell.

Many chatbots are designed to handle the conversation alone for as long as possible, because every handover to a person costs money. The result is a dead end. The customer goes round in a loop of stock answers, with no clear way to reach someone who can help. Few things annoy a customer faster than feeling trapped in a conversation that goes nowhere.

Chatbots are often sold on their resolution rate: the share of questions the bot closes without a person. That number looks good on a dashboard. It doesn't tell you whether the answers were right. A bot that resolves 95% of questions sounds better than one that resolves 75%, even if a large part of that 95% was wrong or useless. The number that matters is how many customers actually got what they needed. That's much harder to put on a sales page.

When a chatbot is a bad idea

An AI chatbot is only as good as the knowledge it draws on. If you have few or no documented answers, the bot has nothing to fall back on, so it guesses. In that case, fix your knowledge problem first and think about a chatbot after that. A bot on top of an empty or messy knowledge base makes things worse.

If your customers' questions are nearly always unique, sensitive or complex, there's little repetition to automate. Think of legal or medical advice, or situations where how the customer feels matters more than the facts. A chatbot adds little there, and a well-trained person is the right choice. Be honest with yourself about that, even when a chatbot looks temptingly cheap.

And sometimes it's simply too early. If you get a handful of questions a week, the time it takes to set up a chatbot properly won't pay off. A clear FAQ or a person who's easy to reach does more for your customers. A chatbot is a tool to save time, and it earns its place once there's enough repetition to actually save that time.

What does work

The bots that do improve customer service have one thing in common: they're built around honesty. They only answer when they have a real basis for the answer, and they're open about what they don't know. That's the approach we chose at Sarrai. In practice, it looks like this.

A good answer comes from your own documentation, with the source shown. The underlying AI model isn't allowed to use its general knowledge of the internet to answer your customers, because that's where hallucinations creep back in. At Sarrai, an answer without a source isn't an answer. With every answer, you see which article it's based on, so you can always check it.

The most important thing an assistant must be able to say is that it doesn't know. We wrote earlier about why 'I don't know' is the most important sentence in an AI system. When the assistant can't find a grounded answer, it says so and hands the conversation to a person, with the full conversation history. The customer knows straight away where they stand, and doesn't get stuck in a loop.

And that person's answer doesn't get lost. As soon as your colleague answers the question, the answer flows back into your knowledge base, so the assistant knows it next time. Your knowledge gets a little better with every conversation, without turning it into a separate writing project. The measure is still how many customers actually got help, and that number goes up as your knowledge grows.

A chatbot is only as good as the knowledge behind it

Putting an AI chatbot on your customer service doesn't improve it by itself. Three things make the difference: the knowledge the assistant draws on, how honestly it deals with what it doesn't know, and how easily a person can take over when needed. Get those right, and your customer service becomes faster and more personal at the same time. Skip them, and you've got a smooth talker that drives your customers away.

Want to read more? Find out what an AI helpdesk is and whether you need one. If you're weighing up a new helpdesk, compare the best Zendesk alternatives. Or see how to automate your customer service without hiring.