The short answer
Response time stops discriminating between good and great once answers are instant. The five measures that still predict revenue are deflection rate, knowledge coverage, escalation quality, after-hours share and conversation-to-order rate - and each one is misleading alone, which is why they are reviewed together monthly.
Most businesses measure support with one number - response time - and it is the least interesting one. It was a good metric when replies took hours and the variance between teams was real. Once an AI agent answers in under two seconds, it stops discriminating between good and great, because a brilliant answer and a useless one arrive equally fast.
Here are five that still discriminate, what each one tells you to fix, and - just as important - how each one gets gamed.
1. Deflection rate: what never needed you
Deflection is the share of conversations resolved without a human ever touching them. Healthy grounded setups sit around 80-90%, and the number is worth watching weekly for the first two months because it moves fast in that window.
If yours is lower, the cause is almost never the AI. It is coverage: the answers your customers want are not in your documents yet. Chase deflection by writing, not by tuning a threshold - a low deflection rate is a reading comprehension test your knowledge base is failing.
How it gets gamed: bury the talk-to-a-person button and deflection climbs immediately. It also means customers who wanted a human could not find one, which shows up two months later as reviews rather than as a metric. Never read deflection without reading escalation quality beside it.
2. Knowledge coverage: the gap list
Every question the agent could not answer is logged. That list is the single most valuable output of the entire system, and it is the most frequently ignored - because reading it is nobody's job unless somebody makes it theirs.
It is an editorial calendar written by your own customers. A gap appearing once is noise. A gap repeating daily is a document you owe them by Friday - and an unwritten one is how a backlog starts compounding. The useful shape is a list that spikes in week one, shrinks steeply through month one, and then settles into a slow trickle of genuinely novel questions.
How it gets gamed: it does not, which is what makes it the most honest number here. What it does instead is get ignored, and a gap list nobody reads produces a knowledge base frozen at whatever your day-one guess was.
3. Escalation quality: the right conversations, not fewer
Do not celebrate a falling escalation count on its own - celebrate what is in it. The escalations that reach you should be genuinely human-worthy: disputes, damaged orders, bulk enquiries, the edge cases no policy anticipated - and a surprising share of them are buying signals rather than problems.
If 'what's your return window?' is still reaching a person, that is a coverage bug wearing an escalation costume, and the fix is a document rather than a process. Read five escalation transcripts a week - not a summary, the actual transcripts. Ten minutes, and it tells you more than any dashboard.
One extra signal in the transcripts: where in the conversation the handoff happened. Escalations on the first message usually mean customers arriving with problems no document can solve, which is healthy. Escalations on the fourth message mean the agent answered three times without resolving anything, and those are the transcripts worth reading closely.
The goal isn't zero escalations. It's zero boring escalations.
4. After-hours share: the demand you were forfeiting
What proportion of your conversations start outside business hours? For most Indian SMBs it is more than half, which is startling the first time it is measured and obvious in hindsight - people shop after dinner, not during their own workday.
Before automation, that share is the demand you were silently forfeiting, because a question asked at 10pm and answered at 9:30am was decided eleven hours before your reply arrived. After automation it becomes the clearest proof the system is doing something, since it is the segment that previously got nothing at all. The full argument is here.
Watch it alongside conversion in the same window. If after-hours conversations convert close to daytime ones, the coverage is working. If they lag badly, the questions being asked at night are ones your documents do not answer.
5. Conversation-to-order rate: the number that closes the loop
Of the customers who started a conversation, what share bought? This is the metric that moves support out of the cost column, and it is the one most small teams have never calculated.
The comparison that matters is against customers who did not converse at all. In most stores the conversation cohort converts noticeably better, which is intuitive - asking a question is a purchase signal - but the size of the gap is the argument for putting the widget on the product page and the cart rather than hiding it on a contact page. Unanswered pre-purchase questions are where this leaks.
How it gets gamed: not deliberately, but it flatters itself. People who ask questions were already more likely to buy, so do not claim the whole gap as causation. The honest version compares the same question asked in hours against out of hours, where the only variable is whether anyone answered.
Reading them together
Any single one of these is misleading. High deflection with a hidden human button is a bad system scoring well. A short gap list with low deflection means the agent is answering confidently from thin documents, which is worse than refusing. A falling escalation count with rising complaints means conversations are being closed rather than resolved.
Together they describe something no individual number can: whether your support is compounding. A system that is compounding shows deflection rising, the gap list shrinking, escalations getting more interesting, after-hours share converting like daytime, and conversation-to-order holding steady as volume grows.
Set a monthly review of thirty minutes with all five in front of you, and read five escalation transcripts before you look at any of the numbers. The transcripts will usually tell you what the metrics are about to say.
| Metric | Healthy direction | What a bad reading means |
|---|---|---|
| Deflection rate | Rising, then plateauing high | Thin documents, not a weak model |
| Knowledge gap list | Spikes, then shrinks | Nobody is reading it |
| Escalation quality | Fewer routine, more real | Coverage gaps reaching a person |
| After-hours share | Converting like daytime | Night questions your documents miss |
| Conversation-to-order | Steady or rising with volume | Answers are fast but not useful |
Frequently asked questions
What is a good deflection rate?
Healthy grounded setups sit around 80-90% of conversations resolved without a human. If yours is materially lower, the cause is almost always coverage - the answers customers want are not in your documents yet - rather than anything about the AI.
Is a falling escalation count always good?
No. What matters is what is in the escalations, not how many. If routine questions your documents already cover are still reaching a person, that is a coverage gap. If only disputes and bulk enquiries reach you, the system is working even if the count is flat.
Why is response time a weak metric?
Because it stops varying. Once every answer arrives in under two seconds, the number is identical for a brilliant answer and a useless one, so it can no longer tell them apart.
How often should these be reviewed?
Monthly, together. Any single one of them can be gamed - deflection by burying the human handoff, escalation count by refusing to escalate - and reading them side by side is what makes the gaming visible.
