Where AI helps and where it absolutely does not
AI for customer support · Lesson 2 / 20
The admission matrix
The usefulness of AI in support is decided by two axes: the volume of the task and the cost of an error. Volume tells you whether the effort pays back. Cost of error tells you whether the model may face a customer without a human in between at all.
Green zone: do it this quarter
- Draft replies for agents. The model writes, the human edits and sends. Saves 30–50% of typing time.
- Classification and routing. Detect topic, language, tone and urgency, then push the ticket into the right queue.
- Summarising long threads. A forty-message ticket collapses into six lines for whoever takes over the shift.
- Knowledge base retrieval. The model finds the right article where keyword search fails.
- Bulk analysis of tickets. Clustering a thousand tickets into twenty themes in one pass.
Red zone: never without a human
- Money. Refunds, compensation, charges, changes to an invoice amount.
- Legally loaded wording. Contract terms, warranty, personal data, consumer rights.
- Account security. Access recovery, ownership transfer, identity verification.
- Promises about timing. When a bug gets fixed, when the courier arrives, when a feature ships.
- Health, finance, law. Any domain where wrong advice causes direct harm.
A topic may be admitted to auto-reply only if all four hold: 1. The answer is deterministic: one question, one correct answer 2. The source of truth exists in the knowledge base or in an API 3. The cost of an error is a re-ask, not lost money 4. On a test set of 100 real questions from that topic, accuracy is at least 95% and confidently wrong answers are exactly zero
Insight. AI in support does not fail on hard questions. It fails when it answers confidently and is wrong. The price of a confident error is always higher than the price of an honest I do not know.
Common mistake. Letting the model into the red zone because that topic is only three percent of volume. Those three percent produce the public complaints, the chargebacks and the disputes, each of which costs more than the entire saving on the green zone.
Pro tip. Put a hard stop-list of topics into the classifier, before generation. If a ticket matches the stop-list it goes straight to a human and the model never drafts anything, so no agent is tempted to send the draft as it stands.
Cheat sheet
- Two axes decide everything: topic volume and cost of error.
- Drafts, classification, summaries and retrieval are green.
- Money, law, access and timing promises stay with humans.
- The stop-list runs before generation, not after sending.
1. Which of these belongs in the red zone for auto-replies?
2. Why is a confident error worse than refusing to answer?
3. At what point should the topic stop-list fire?
Task — checked by AI
Take your own ninety-day export and place your top fifteen topics on the matrix: monthly ticket volume against cost of error. For each topic state the zone (green, amber, red) and one line of reasoning. Separately write out the stop-list of topics that will never be automated under any circumstances, and name what specifically happens to the customer if the model gets each of them wrong.