AI Hallucination

An AI hallucination is output from a generative model that is presented confidently but is wrong, made up, or not supported by the input it was given. A large language model might invent a statistic, a citation, a product feature or an entity that never appears in the text it was asked to analyse.

How it works

LLMs generate text by predicting likely next tokens. They are optimised to produce fluent, plausible language and have no built-in step that checks facts against a source of truth. Research from OpenAI in 2025 argues that training and benchmark scoring also reward guessing over admitting uncertainty, so models learn to answer confidently when they should say “I don’t know”. Hallucinations become more likely when the model lacks the relevant information, when the prompt is ambiguous, or when the input is truncated, for example when a page is too long for the model’s context window.

Common mitigations are grounding the model in retrieved documents (RAG), structured sources such as knowledge graphs, tighter prompts that tell the model to use only the supplied text, and checks that match outputs back to the source.

How it’s used in SEO and marketing

Hallucination matters in two directions:

• In your own workflows. Meta descriptions with invented numbers (“trusted by 10,000 marketers”), alt text that misdescribes an image, or entity extraction that returns entities not in the text.
• About your brand in AI search. Assistants may describe products you do not sell or attribute the wrong facts to you, especially when information about your entity is thin or inconsistent across the web.

Example

In MLforSEO’s comparison of NLP APIs and generative AI for entity extraction, some chatbots returned entities not found verbatim in the text and expanded acronyms on their own. In a workflow that matches entities back to pages, an entity that is not in the text is a hallucination, however plausible it sounds. The purpose-built NLP APIs made different mistakes, but did not invent entities.

Limitations and things to watch

• Hallucinations are hard to spot without the source. Ask the model for quotes or timestamps as evidence, then spot-check them.
• Lowering temperature reduces randomness but does not remove hallucination, and many newer models no longer accept a temperature setting.
• A model’s stated confidence is not a reliable signal of correctness.
• Keep a human review step before anything generated is published.
• For your brand, consistent entity data across your site, schema and external profiles reduces the gaps that models fill with guesses.

Learn it on MLforSEO

• Comparing NLP APIs for Entity and Sentiment Analysis (vs Generative AI)
• How to Automatically Generate Meta Descriptions with Python: From BERT to LLMs
• Image SEO in 2026 covers hallucinated alt text.
• AI Search & LLMs: Entity SEO and Knowledge Graph Strategies for Brands course, bonus lesson comparing NLP APIs with generative AI.