What grounding means for an AI support agent

A language model given a question about your return policy will produce an answer about a return policy. Fluent, confident, correctly formatted, plausible. Whether it is your policy is a separate question, and the model has no way to raise it.

That gap is the whole reason grounding exists, and it is the single most useful thing to understand before putting an AI in front of customers.

The default behaviour is the problem

A model trained on a large amount of text has absorbed how return policies are usually written. Asked about yours, it draws on that: a plausible window, plausible conditions, plausible wording. The output is not a guess presented as a guess. It is a statement in your brand's voice, sent from your business's number, and the customer has no reason to doubt it.

This is what a hallucination is in this setting: not an amusing error, but a written commitment your business now has to honour or retract. An invented delivery date is a promise. An invented refund window is a policy. In a messaging channel it is worse than on a help page, because it is a one-to-one statement to a named person who kept the receipt.

What grounding actually does

Grounding means the answer is produced from source material you supplied, rather than from what the model absorbed in training.

In practice it works in three steps.

1. Source. Your policies, product details, delivery information, past answers, and the customer's own records are collected and indexed.
2. Retrieval. When a message arrives, the system finds the passages relevant to it.
3. Constrained generation. The model writes an answer using those passages, and is instructed to work from them rather than around them.

Done properly, an answer about your return policy is a restatement of the paragraph you wrote about your return policy. Done properly, it also cites which paragraph.

The real test is the empty case

Any grounded system looks good when retrieval works. What separates one worth trusting is what happens when retrieval finds nothing.

There are two possible behaviours and they are not close in value.

It says so. "I do not have that information. Let me get someone who does." The thread is passed to a person with the full context. The customer waits a little longer and gets a true answer.

It answers anyway. It has an instruction to be helpful, a question it cannot source, and a model that is very good at producing text. So it produces text.

When you evaluate a product, this is the thing to test. Ask it something your material genuinely does not cover, phrased so it sounds like a normal customer question. A system that admits the gap is a system you can extend. A system that fills the gap will do so in production, on a topic you did not think to check.

Grounding is not fine-tuning, and neither is "trained on your data"

These get used interchangeably in sales conversations and they mean different things.

Fine-tuning adjusts a model's behaviour with examples. It is good for tone, format, and the shape of a response. It is a poor mechanism for facts, because facts change on a Tuesday and re-training does not happen on a Tuesday.

Grounding puts the facts outside the model, in material you can edit. Change the paragraph, and the next answer changes. That property is what makes it maintainable.

"Trained on your data" is ambiguous by construction. The question to ask is direct: when I change a sentence in my policy document, when does the next answer change? A grounded system answers "immediately, or as soon as it re-indexes". Anything vaguer is describing something else.

Two kinds of source material, and both are needed

Most discussion of grounding covers documents. Customer messaging needs a second kind.

Document grounding answers general questions: what is the policy, how does the warranty work, do you ship there, what are the sizes.

Record grounding answers the specific ones: where is my order, when does my appointment start, what did I pay. These come from a system of record rather than from a document, and they are the questions customers actually ask most.

A product that handles the first and not the second will look impressive in a demo and deflect very little real volume, because "where is my order" is the message that arrives forty times a day. When you evaluate, ask what the AI can read from your order system, and what happens to a question it cannot look up.

Why the knowledge base is the whole ballgame

An AI cannot answer better than the material it was given. That sounds obvious and it has an unpleasant corollary: most of the work in a good deployment is content work, not AI work.

The common failure is loading everything once and never revisiting it. Six months later the system is confidently quoting a policy that changed, and doing it well, in your voice, at scale. Grounding made the wrong answer more convincing rather than less.

Two habits fix this, and neither is technical.

Give the content an owner. Somebody whose job includes the source material being current. Treat it like a page on your site, not like a one-time upload.

Look at what had no source. Every question where retrieval found nothing is a gap in your material and a note about what customers want to know. A product that reports those is telling you what to write next. That list is usually the highest-value output of the whole system in the first months.

What a grounded answer should show you

Grounding is a claim, and claims that cannot be checked are worth little. Three things make it verifiable.

Named sources on the draft. Which passages the answer used, visible to the person reviewing it, before it sends. A source list turns "does this sound right" into "is this the right paragraph".

An honest confidence signal, wired to something. Confidence is only useful if a low score changes behaviour: handing off to a person, holding for approval, or asking a clarifying question. A number displayed next to a message that sends regardless is decoration.

A record of what was sent. An audit trail covering what the AI drafted, who approved it, and what actually went out. When a customer quotes something back at you, this is what tells you whether the system said it.

Where automation should stop

Grounding reduces the risk of a wrong answer. It does not make an answer safe to send unsupervised in every case.

The categories worth an approval gate are consistent across most businesses: anything involving money, anything involving a commitment about a date, anything about a complaint, and anything the system is unsure of. The detail that makes a gate real is binding: the approval should commit to the exact text that sends, so nothing can change between the review and the delivery.

Routine informational replies, grounded and sourced, are the volume worth automating. That is a large share of inbound messages and it is enough.

What to ask a vendor

• Show me an answer with its sources. Which specific passages did it use?
• What happens when retrieval finds nothing? Show me, on a question my content does not cover.
• Can it read my order records, or only my documents?
• When I edit a policy, when does the answer change?
• What is held for approval, and does the approval bind the exact text?
• Can you show me the questions it could not answer last week?

A product built around grounding answers all six with a screen. A product that does retrieval as a feature will answer the first and get vague from the second onward.