AI for customer support3 min read
What grounding means for an AI support agent
Why an ungrounded answer is risky
Without your policy, a model fills the gap with patterns from other businesses: a typical return window, standard exclusions. Sent from your number, the customer reads it as your policy. An invented delivery date or refund window creates a promise your team must then undo.
How grounding works
- Source. Your policies, product details and customer records are collected and indexed.
- Retrieval. When a message arrives, the system finds the relevant passages.
- Constrained generation. The model writes the answer from those passages.
A citation is where checking starts. Confirm that the passage supports the answer and applies to this customer.
The real test: a missing answer
Ask a normal question your material does not cover. If your returns page ignores clearance items, ask whether a clearance item can be returned.
A good reply names what is missing and gives a next step: it asks for a detail or passes the question to someone who can decide. It never invents a deadline or promises an exception.
Then ask a related question your material does answer, to check it still finds real evidence.
Grounding is not fine-tuning
| Term | What it does | What to ask |
|---|---|---|
| Fine-tuning | Shapes tone, format and task behavior | How do changed facts reach the answer? |
| Grounding | Answers from evidence kept outside the model | How long after I edit a source does the answer change? |
| "Trained on your data" | Says nothing about updates | Show me: edit a source, wait, ask again. |
Two kinds of source
- Documents answer general questions: the policy, the warranty, delivery areas, sizes.
- Records answer customer-specific ones: my appointment, my viewing, my invoice or order. These need the right record and permission to share it.
Ask how the integration identifies the customer, and what happens when a record is missing, stale or someone else's. A plausible answer is not proof that a lookup succeeded.
Keep the sources current
The common failure is loading everything once and never revisiting it. Six months later the agent quotes an old policy, fluently and in your voice. Keep your knowledge base current:
- Give the content an owner. Treat it like a page on your site, not a one-time upload.
- Sort unanswered questions by cause. Add content when information is missing; fix retrieval or the integration when it exists but did not reach the agent.
- Respect access limits. Never bypass a restriction just to produce an answer.
What a grounded answer should show
- Your approved content behind every reply, so a wrong answer points to a page you can fix.
- A defined stop. When evidence is missing, the agent asks a question or passes the chat to a person. It does not guess.
- A record of what was sent. An activity history lets your team trace a reply a customer quotes back.
When Otobiz's AI reply agent lacks the answer, it asks one question or says the team will check, and after a second miss a person takes the chat. Refunds, payment disputes, legal and medical questions go straight to your team.
Questions for a vendor
- Show me an answer from our own content, then one it does not cover.
- Show me what happens when retrieval finds nothing.
- Can it read the appointment, invoice or order records this workflow needs, with the right access?
- When I edit a policy, when does the answer change?
- Which questions could it not answer last week?
Record which answers were shown, which were only described, and which remain untested.
Test replies before customers see them
Write down the correct answer for a few real questions first, and keep personal data out of the test sheet.
| Test | Pass when |
|---|---|
| A current price or policy | It matches the current source |
| A fact missing from your sources | It asks, or admits the gap |
| Two conflicting policy pages | It does not invent a compromise |
| A follow-up like “what about tomorrow?” | It keeps context from earlier in the chat |
| A request to refund or book | It separates a request from a completed action |
| A request for a person | It gives the customer a useful next step |
Download the reply-quality worksheet. Log whether each failure came from missing information, retrieval, reasoning or the next action, fix that cause, and rerun every question.
A small sheet is a start, not proof that every conversation is safe to automate. To see a finished conversation first, explore a customer journey.
NIST's Generative AI Profile calls confident false output "confabulation" and sets out a neutral framework for managing it.
