See what I did there? 😉 You will. When it comes to large language models (LLMs), the probability that hallucinations will go away is zero.

I referred to this when covering the AI Card Game created by Aligned Discovery and Tara Emory a couple of days ago, where Tara discussed how hallucinations actually happen. But since that was toward the bottom of the post and since I have recently heard a couple of people mention the idea that hallucinations will eventually go away, I thought this topic merited its own separate post.

People think of LLMs (like ChatGPT, Claude, and Gemini) as knowledge bases, but they are far from that. LLMs are designed to predict language, not verify truth. That distinction is very important in understanding why these systems sometimes produce hallucinations, where the responses that sound convincing but contain inaccurate, misleading, or completely fabricated information.

As Tara and I discussed in our walk through of the AI Card Game, LLMs work by predicting the most likely next word in a sequence based on patterns learned from massive amounts of text. During training, the models analyze not billions and billions – but literally trillions of words from books, articles, websites, and other sources to learn how language is structured and how ideas are commonly connected. When a user enters a prompt, the model does not “look up” an answer in the way a search engine does. Instead, it generates a response one word at a time by calculating probabilities about what should come next.

That’s right, LLMs are fundamentally probability-based prediction systems. They are optimized to produce language that sounds natural and plausible, not necessarily language that is guaranteed to be true. That design is the primary reason hallucinations occur.

For example, if a user asks about a court case, the model may recognize the patterns of legal writing and generate something that resembles a real judicial opinion or legal citation. But if the model lacks reliable information about the actual case, it may still produce an answer because its objective is to continue the pattern of language. In some instances, this results in invented case names, fabricated quotations, or inaccurate legal holdings that appear authentic on the surface. It’s happened at least over 1,500 times in the last three years.

Hallucinations happen for several reasons. One major factor is the quality and structure of the training data. LLMs are trained on enormous collections of publicly available and licensed text, much of which contain conflicting information, outdated facts, mistakes, opinions, satire, or fictional material. The model learns patterns from all of it. While developers attempt to filter and refine training data, no dataset is perfect.

But in many cases, you can’t blame the data. The primary factor is that the model does not truly “understand” information the way humans do. Humans rely on reasoning, experience, and external validation to determine whether something is true. LLMs don’t possess that kind of grounded understanding. Instead, they rely on statistical relationships between words and concepts. If those statistical relationships suggest that a certain response fits the prompt, the model may generate it even when the information is wrong, because – rather than refusing to answer – the model may attempt to fill the gaps with language that appears reasonable.

As a result, hallucinations aren’t simply software “bugs” that developers can patch away completely. They’re a natural consequence of how generative AI systems function. The same probabilistic prediction process that allows an LLM to write essays, summarize documents, generate code, or draft creative content also creates the possibility that it will produce false information.

Sure, improvements to the models over the years have significantly reduced hallucinations. Modern models are generally far more reliable than earlier versions. However, even advanced systems connected to authoritative data sources can still make mistakes. A model can misinterpret information, combine accurate and inaccurate details together, or draw unsupported conclusions from retrieved material. That will never change.

That’s why as long as LLMs primarily operate through probability-based prediction, the probability that hallucinations will go away is zero. Now, you see what I did there! 😁

At least I hope you do, because that understanding is key to setting expectations as to what LLM models can and can’t do. Hallucinations will always be a part of their probabilistic nature – they’re not going away, not at least for LLMs. That’s what Tara and I discussed, but since many of you may have missed it, I’m reinforcing the message.

So, what do you think? Do you agree that the probability that hallucinations will go away is zero? Please share any comments you might have or if you’d like to know more about a particular topic.

Image created using DALL-E 3, using the term “robot lawyer wearing a suit expressing shock when looking at a workstation”.

Disclaimer: The views represented herein are exclusively the views of the author, and do not necessarily represent the views held by my employer, my partners or my clients. eDiscovery Today is made available solely for educational purposes to provide general information about general eDiscovery principles and not to provide specific legal advice applicable to any particular circumstance. eDiscovery Today should not be used as a substitute for competent legal advice from a lawyer you have retained and who has agreed to represent you.