Hallucination
A hallucination is when an AI model states something false as if it were fact, often fluently and with no source behind it.
A hallucination is when an AI model states something that isn't true as if it were fact. The answer reads well and sounds sure of itself, but the claim is made up: no page says it, or the model has stitched two true things into one false one.
For marketers, hallucinations are the reason you can't assume AI answers about your brand are right just because they sound right.
Why models hallucinate
A language model writes by predicting likely next words. Most of the time that produces true sentences, because true sentences are common in what it learned. But when it doesn't know something, it doesn't always say so. It writes the likeliest-sounding answer instead.
For example, asked when Harrow & Pine was founded, a model with nothing to go on might say it was founded in 2012, because that's a plausible year for a brand like it. Nothing in its training said 2012.
When it's more likely
Hallucinations come up more often when:
- The model answers from memory, without searching
- The subject is small or new, so there's little written about it
- The question asks for specifics, like dates, prices, numbers or names
- The question assumes something false, like "Why did Kestrel Roasters close?" when it didn't
Grounding helps. When a platform reads current pages before answering, it has something to go on. But it can still misread a page or mix up two of them.
Hallucination and your brand
A hallucination about your category is annoying. A hallucination about your brand can cost you customers who never find out it was wrong. That's why it's worth checking what the platforms say about you against what's true. In AnswerOrbit, Inaccuracy drivers in Claims shows the pages behind each wrong claim, or the platform's own knowledge when no page was used.