The short answer
When a grounded AI support rollout disappoints, the cause is almost never the model. It is thin or contradictory documents, a hidden escalation route, nobody reading the knowledge-gap list, or content that went stale. Each has a fix measured in hours, and the gap list tells you which one you have.
The failure mode of an AI support rollout is rarely dramatic. Nothing breaks. The bot just answers less well than the demo suggested, escalates more than expected, and after a month somebody quietly stops talking about it. Every time we have looked into one of these, the cause was on this list - and none of the seven are about the model.
1. The knowledge base is thinner than the inbox
This is the single most common cause, and it presents as the bot being bad. Someone uploads three documents, asks fifteen questions, gets eight good answers and concludes the technology is unreliable.
A grounded agent answers only from what you give it. If the answer is not in a document, refusing is the correct behaviour - the alternative is inventing your return policy, which is much worse and much harder to detect. So a high refusal rate is not a quality signal about the AI; it is a coverage report on your documents.
The fix is the export. Take a month of real support messages, sort by question, and write the answers to your top fifteen. That is an afternoon and it moves the deflection rate more than any setting will.
2. Two documents disagree
The second most common, and the most confusing to debug, because the bot appears to be inconsistent. It is not - it is faithfully quoting whichever of your two contradictory files retrieval reached first.
It happens innocently. The return window is in your FAQ, in your terms page, and in a policy PDF from last year. Two of the three say seven days and one says fourteen, because one was updated and the others were not. Whichever one surfaces is the one the customer gets, and it is cited, which makes the wrong answer look authoritative.
One fact, one place. When a duplicate turns up, delete it rather than editing both - a fact that lives in two files will disagree again within a quarter. The knowledge-base guide covers the structure that prevents this.
The bot is not inconsistent. It is being consistent with a knowledge base that contradicts itself.
3. The route to a human is buried
Some teams hide the talk-to-a-person option, on the theory that visible escape hatches reduce deflection. It reads as confidence and lands as hostage-taking, and it produces the screenshots that end up on social media.
The counter-intuitive part is that a visible button gets pressed less, not more. Knowing the exit exists is usually enough for a customer to try the bot first. The ones who press it immediately were never going to accept an automated answer, and you want those people in your inbox rather than in your reviews.
Keep it visible from the first message, and make sure what travels across it is the whole conversation - the escalation post covers what has to be attached.
4. Nobody reads the gap list
Every question the agent could not answer is logged. That list is the single most valuable output of the whole system and it is the most frequently ignored, because reading it is nobody's job by default.
It is an editorial calendar written by your customers. A gap that appears once is noise; a gap that appears daily is a document you owe them by Friday. Teams that read it weekly see deflection climb for the first two months and then plateau at a genuinely high number. Teams that do not read it stay wherever their day-one guess landed them.
Twenty minutes a week, on the calendar, with a name against it.
5. The documents went stale
This one arrives later, usually around month four, and it is the failure mode that grounding does not protect you from. A grounded agent quotes your documents accurately. If the document is out of date, it quotes the out-of-date policy - confidently, with a citation, which makes it more convincing than a guess would have been.
The classic case is a festival price sheet still uploaded in January, or sale terms that expired in November. The bot is behaving perfectly and telling customers something that is no longer true.
Twenty minutes a month: reread the two documents with numbers in them, delete anything about an offer that has ended, and check that the filename still tells you when it was written. Dating filenames - fees-aug-2026.pdf - turns that review into a scan instead of a reading exercise.
6. The widget is in the wrong place
A support agent installed only on a contact page will look ineffective, for the same reason an FAQ page does not deflect on its own: the questions that matter are not asked there. They are asked on the product page, where sizing and delivery doubts occur, and in the cart, where an unanswered doubt becomes an abandonment.
Leave it everywhere. The widget is under 50KB and loads asynchronously, so there is no page-speed argument for rationing it - and if you are on a metered plan where every conversation costs, note that rationing the widget is exactly the behaviour that pricing model produces.
7. There was no baseline
The quietest failure: the rollout works and nobody can tell, because nothing was measured beforehand.
Write down four numbers before you launch. How many support messages arrive in a week. What share arrive outside business hours. How long your median first reply takes. And roughly what fraction of messages are your top fifteen questions. None of that takes more than an hour to establish, and without it every later conversation about value is an argument about impressions.
A month in, the same four numbers tell you plainly whether this worked. The metrics post covers what to track once the obvious ones stop being interesting.
The fortnight that decides it
Almost all of this resolves in the first two weeks, and the pattern is consistent: week one exposes the distance between the questions you predicted and the questions customers actually ask, and closing that distance is what makes week three look good. Teams that expect the gap and work it get a system that keeps improving. Teams that expect day-one perfection conclude the technology does not work and go back to answering everything by hand.
Start with the export, write the five documents, keep the button visible, and put the gap review on the calendar. The free plan is enough to run the whole fortnight before spending anything.
Frequently asked questions
My bot says 'I don't know' too often. What is wrong?
Almost always coverage, not configuration. Read the knowledge-gap list: if the questions it could not answer are ones your documents genuinely do not address, the fix is writing, not tuning a threshold.
Why does the bot give a different answer than my website?
Two documents disagree. When the same fact appears in more than one uploaded file, retrieval can surface either one. Keep a single source of truth per fact and delete the duplicate.
How long before an AI support rollout is working properly?
Expect a fortnight. Week one exposes the gap between the questions you predicted and the ones customers actually ask; closing those gaps is what makes week three look good.
Should I tune the confidence threshold on day one?
No. You have no data yet, only nerves. Run the default for two weeks, read the transcripts, then change one thing and watch another fortnight.
