Hallucination

In AI, a hallucination is generated content that presents a false or unsupported detail as fact. It can be as small as a time added to an otherwise accurate summary.

Which detail did the model add?

Imagine asking a model to summarize this fictional library notice.

Source notice

“The library’s map room will reopen on Saturday. Please use the east entrance.”

Model’s summary

“The map room reopens Saturday at 9 a.m. Visitors should use the east entrance.”

Check the answer one claim at a time

The notice supports the day and the entrance. It gives no opening time. Nine o’clock might turn out to be correct, but the summary has introduced a detail the source cannot establish. A faithful answer would say that the room reopens on Saturday and the notice gives no hour.

The same check works for a figure, a name, or a quotation. Find the passage that supports that exact detail. If an answer cites a report, check that the report exists and says what the answer attributes to it.

Why fluent answers can contain invented details

A language model generates text from learned patterns and the context it receives. An opening time fits the familiar wording of a reopening notice. Generating that phrase requires no independent confirmation of the schedule.

Retrieval-augmented generation can bring the actual notice into the model’s context. Asking for the supporting passage makes the claim easier to check. Guardrails can flag a missing citation or route an uncertain answer for review. The decisive check here is small enough for a reader to do directly: look for an opening time in the notice.

Follow the idea

Grounding shows how to tie a claim to evidence.

Retrieval-augmented generation explains how sources reach a model before it answers.

Source · Maynez and colleagues on faithfulness in generated summaries

Return to Hallucination in the A–Z glossary