AI chatbots make things up because they are built to write text that sounds likely, and nothing in that process checks the text against the world. When the model does not know a fact, it still produces the most plausible-sounding answer. Training then makes things worse in one way: models are mostly scored on how often they are right, so a confident guess earns more than an honest “I don’t know”.
What is an AI hallucination?
An AI hallucination is an answer that sounds confident and fluent but is false. OpenAI defines hallucinations as “plausible but false statements generated by language models”. The made-up thing can be a date, a quotation, a book title, a court case or a company policy.
The word “hallucination” is a little misleading, because the model is not seeing anything that is not there. Some researchers prefer “confabulation”, the word doctors use when a patient fills a gap in memory with a story they believe. Either way, the result is the same: an invented answer delivered in the same calm tone as a correct one.
Why does it happen?
A chatbot such as ChatGPT writes one word at a time, each time picking a likely next word given everything written so far. We explained this in How Does ChatGPT Decide What to Say?
This works very well for anything that follows patterns. Grammar, spelling and the general shape of an explanation repeat across millions of texts, so the model learns them. Rare, one-off facts are different. OpenAI’s researchers give the example of a pet’s birthday: it “cannot be predicted from patterns alone”. If the model has seen a fact only once or twice in training, or not at all, it has nothing reliable to go on. It still has to write the next word, so it writes one that fits.
They tested this on one of their own authors, Adam Tauman Kalai. Asked for the title of his PhD dissertation, a widely used chatbot gave three different answers, all wrong. Asked for his birthday, it gave three different dates, also all wrong.
Why doesn’t it just say “I don’t know”?
Partly because of how models are graded. In a paper published in September 2025, OpenAI compared it to a multiple-choice exam: “If you do not know the answer but take a wild guess, you might get lucky and be right. Leaving it blank guarantees a zero.”
Most of the tests used to compare AI models score only accuracy. A model that guesses a birthday has a 1 in 365 chance of being right. A model that says “I don’t know” scores nothing. Over thousands of questions, the guesser looks better on the leaderboard, and developers build towards the leaderboard. OpenAI’s own figures show how big the difference can be. On one factual test, its o4-mini model almost never declined to answer, and got 24% right and 75% wrong. Its gpt-5-thinking-mini model declined about half the time, and got 22% right and only 26% wrong. On an accuracy-only leaderboard the first model looks slightly better.
What does cybernetics say about this?
In cybernetics, a system stays on course through feedback: it compares what it did with what it should have done, and corrects the difference. A thermostat checks the actual temperature of the room. Without that check it would simply run the heating and hope.
A chatbot writing an answer has no such check. Its only feedback, while it writes, is its own text. Each new word is chosen to fit the words before it, and nothing compares it with the world. So the loop keeps the answer fluent and consistent, and it can stay consistent with a mistake for a whole page.
The training loop has a problem of its own. A feedback loop teaches whatever it rewards. If the scoring rewards a lucky guess as much as real knowledge, the model learns to guess. This is the same pattern we saw in the sycophancy spiral, where rewarding agreeable answers taught a model to flatter.
The cybernetician W. Ross Ashby put the general rule in four words: “only variety can destroy variety” (his law of requisite variety). The world holds far more particular facts than any model can store. Where the model’s store runs out, something outside it has to do the checking.
What happens when nobody checks?
In February 2024, a Canadian tribunal ruled against Air Canada after its website chatbot told a customer, Jake Moffatt, that he could claim a bereavement discount after he had travelled. The airline’s real policy said otherwise. Air Canada argued that the chatbot was responsible for its own words. The tribunal called this “a remarkable submission” and said the airline was responsible for everything on its website, “whether the information comes from a static page or a chatbot”. Air Canada had to pay him C$812.02.
A year earlier, in June 2023, a New York judge fined the lawyers $5,000 in the case Mata v. Avianca for filing a brief with court cases that ChatGPT had invented. In both stories, nobody checked the machine’s answer against the source before relying on it.
How can we reduce it?
The fixes that work all add the missing check.
- Give the model sources. When a chatbot searches the web or reads a document we give it, it can compare its answer with real text. This reduces made-up facts, although the model can still misread a source.
- Reward honesty in training. OpenAI’s proposal is to penalise confident errors more than uncertainty, and give partial credit for saying “I don’t know”.
- Check what matters. For names, numbers, quotations and legal or medical facts, follow the link or look it up. The person reading is often the last part of the loop.
Common questions
Will AI ever stop making things up? Probably not completely. A model that predicts likely text will always meet facts it has not learned. The rate can go down a lot, though, especially when models can look things up and are trained to admit uncertainty.
Are newer models better? Generally yes, but not evenly. Models with web search make fewer factual errors on current events, and models trained to decline when unsure make fewer confident mistakes. A newer model is not automatically more careful, so it is worth checking how each one behaves.
Why does it sound so sure when it is wrong? Because the tone comes from the same patterns as the words. Confident, well-organised writing is common in the text the model learned from, so that is the style it produces, whether or not the facts underneath are right.
Is a hallucination the same as lying? No. Lying means knowing the truth and saying something else. The model has no separate record of the truth to compare against. It produces the likeliest text, and sometimes the likeliest text is false.
How can I spot a made-up answer? Be most careful with specific details: exact quotes, dates, statistics, references and names of cases or papers. Ask the chatbot for its sources and open them. If it cannot point to a real source, treat the detail as unchecked.
For more on how a loop steers whatever sits inside it, see our guide to feedback and our explainer What Is an AI Agent?