Skip to content
BENIBIX Learn

LLM Hallucinations

Level 5 · Where LLMs go wrong

Why a model states false facts in a calm, sure tone, and where it happens most. How old facts and eager agreement add to it.

Course contentsLesson 14 of 15

After this lesson you can

  • Explain why models make things up
  • Spot the places it happens most
  • Ask questions that avoid eager agreement

12 mincovers 3 glossary entries

Worth reading first: Post-Training and RLHF

Why a model makes things up

A hallucination is when a model states something false as if it were true. It might invent a quote, a book, a court case or a date, in the same calm tone as everything else. In 2023, two New York lawyers were fined $5,000 for filing court cases a chatbot had made up.

It happens because of how the model works. It writes what is likely, not what is checked. When it knows a fact well, likely and true are the same thing. When it does not, it still writes something that sounds right, because training rewarded a sure answer over 'I do not know'.

Made-up facts cluster in certain places: exact numbers, dates, names of small things, quotes, web links and references. Common facts are safer than rare ones. Asking for a source does not help on its own, because the model can invent the source too.

The best fix is to give the model the facts. Put the document, the data or the search results in the prompt, and ask it to answer only from them. Tell it that 'I do not know' is a fine answer. Then check anything that matters against the original.

The knowledge cutoff is the date the model's training text ends. The model knows nothing after it, unless the app gives it new facts. Ask about last month's news and it may describe an older event, or guess. A model often does not know its own cutoff exactly.

Sycophancy is the habit of telling people what they want to hear. Push back on a right answer and the model may switch to a wrong one. Share your view first and it tends to agree. In April 2025, OpenAI rolled back a ChatGPT update because it had become too eager to please.

It comes from post-training. People tend to rate agreeable answers higher, and the model learns that. To work around it, ask neutral questions and ask for the case against your idea. Never take a model's change of mind as proof that you were right.

The made-up line looks just like the true ones. Nothing in the tone gives it away.

Worked example

Jack checks a list of studies

Jack asks a model for five studies on remote work, with links. Three are real. Two have real-sounding titles, real authors and links that go nowhere.

He now pastes the papers in himself and asks the model to work only from those. Every claim in the answer points back to a page he can open.

The words for this lesson3

Check yourself

Why does an LLM sometimes state false facts with confidence?

Finished this lesson?

Marking it complete also marks its 3 glossary entries learned, which updates your roadmap and your saved page. 0 are already marked.