AI Hallucinations: Why Models Make Things Up and How to Spot It

Ask ChatGPT for a court citation and it might hand you one that doesn’t exist. Ask Gemini for a chemical formula and it might invent a compound that was never discovered. This isn’t a bug waiting to be patched — it’s baked into how large language models work. UK businesses now lean on AI for research, legal drafting, customer support and financial guidance. When I looked into this properly, the gap between how confident these tools sound and how often they’re actually right turned out far wider than most people assume.

What Actually Happens Inside a Hallucinating Model

A large language model doesn’t store facts in a database it can check against. It predicts the next word based on statistical patterns learned from billions of documents. Most of the time that process produces something accurate. Sometimes the most statistically likely answer is simply wrong, and the model states it anyway, with zero hesitation.

There’s no internal alarm bell built into the architecture. The system has no genuine concept of “I don’t know” the way a person does. It generates fluent, confident-sounding text regardless of whether the underlying claim is solid or completely fabricated. That mismatch between tone and accuracy is exactly where the danger sits.

Researchers call this a hallucination rather than a lie, because the model isn’t choosing to deceive anyone. It’s producing the most plausible continuation of text based on patterns, and plausible sentences aren’t automatically true sentences. Understanding that distinction changes how you should treat every AI-generated fact.

Why Training Rewards Guessing Over Honesty

OpenAI published research in 2026 explaining why hallucinations persist even in the most advanced models available. Standard training and evaluation methods reward confident guessing over admitting uncertainty. A model that says “I’m not sure” scores worse on most benchmarks than one that guesses and occasionally gets lucky.

Falls apart fast once you see the incentive clearly. Test-takers who guess on multiple choice exams tend to outperform test-takers who leave answers blank, even when guessing carries real risk. Model training runs on almost identical logic, repeated across millions of examples during fine-tuning.

Nobody built this deliberately. It’s an emergent side effect of how reinforcement learning scores model outputs. Fixing it means redesigning the reward signal itself — rewarding calibrated uncertainty rather than just correctness — and that work is still in progress across every major AI lab.

The Numbers: How Often Do Top Models Get It Wrong

Hallucination rates vary wildly depending on the model and the task you throw at it. Google’s Gemini-2.0-Flash-001 held a hallucination rate of just 0.7% as of April 2025, genuinely low for a general-purpose model. Meanwhile TII’s Falcon-7B-Instruct hallucinated in almost one of every three responses, clocking in at 29.9%.

Depending on the prompting method used, some studies put overall LLM hallucination rates anywhere between 50% and 82%. That’s an enormous range, and it depends heavily on how open-ended the question is. Ask a narrow factual question and rates drop; ask for an obscure citation and rates climb fast.

Top frontier models now fabricate facts under 1% of the time on straightforward, well-scoped questions. That’s a sharp drop from the 15–20% rates seen just two years earlier. Progress is real and measurable. The floor still isn’t zero, and it likely never will be with current architectures.

Fake Citations, Invented Stats and Other Hallucination Types

Hallucinations don’t all look the same, which is part of what makes them hard to catch. Some are fake academic citations that read perfectly plausibly — a real-sounding journal, a real-sounding author, a page number that goes nowhere. Others are invented statistics dressed up with false precision, because a specific-looking percentage sounds more convincing than a vague estimate.

I’ve seen this pattern with three different AI writing tools now: ask for a source, get a URL that leads to a 404 page or an entirely unrelated article. Legal and medical claims sit in the riskiest category of all. A hallucinated drug interaction or a fabricated case precedent doesn’t just look embarrassing — it can cause genuine harm if nobody checks it before it’s acted on.

Wrong summaries are the quietest version of this problem. Ask an AI to summarise a long report and it will occasionally invent a conclusion the report never reached. Nobody notices unless they go back and read the source themselves, which most people skip under time pressure.

Real Cases Where Hallucinations Caused Real Harm

US lawyers have already been sanctioned by courts for filing briefs stuffed with fabricated case law generated by AI drafting tools. UK solicitors face the same exposure now that AI-assisted drafting has crept into everyday legal practice. The Solicitors Regulation Authority has warned firms to treat AI output as a draft, never a finished citation.

Client-facing chatbots have quoted wrong refund policies, wrong prices and wrong delivery windows to real customers. One misquoted return policy can cost a retailer far more in goodwill than the AI tool ever saved in support costs. None of this needs malice on the model’s part.