GenAI Challenges Series - Hallucinations
A series of takes on some of the challenges for (and to) Artificial Intelligence
Generative AI Challenges Series - Hallucinations
In this (long-delayed) third installment of the series on Generative AI, I wanted to cover one of the challenges that cast a shadow over the future of AI. Of course, it’s not the only one but it’s pervasive and frequently prevents us from getting the full value of AI. This is the case of hallucinations in generative AI content.
(In further posts, I may cover some of the other ones I find fascinating and critical to discuss, like the case of intellectual property and AI, the challenge of faithful AI detection, the disruption to the job market, the environmental impact of AI, the wild pursuit of AGI, or the obsessive focus -and investment- on AI over other important matters.)
Hallucinations: When the Model Gets (too) Creative
You entered your prompt, and got a thorough response, and it includes some data to support it. Impressive! However, when you look a bit closer, something is odd. No, the James Webb Space Telescope (JWST) did NOT take the very first pictures of an exoplanet outside our solar system. And NO, those cases that ChatGPT quoted to support your legal research don’t even exist.
What happened? How could it get it so wrong? Is it lying? Is it malfunctioning? Should I turn it off and on again? Could I ever trust generative AI again? Should I?
It’s a good thing that you caught it, but what if you hadn’t? Imagine what could have happened if you had set up a chatbot for your auto dealership and it offered a client a discount of $58,000 on a car?
You might have heard that sometimes AI models like GPT can produce content that sounds convincing but isn’t actually true. These moments are called hallucinations—when the model generates information that isn’t grounded in the data it was trained on.
What Are Hallucinations?
In the context of AI, hallucinations occur when the model confidently generates text that seems plausible but is actually incorrect, misleading, or entirely fabricated. For example, the model might generate a fact about a historical event that never actually happened, misattribute a quote, it might contradict itself, or give you the wrong answer to a mathematical operation.
In fact, according to a recent Apple report and Tech Crunch articles, LLMs struggle with small variations in the language used to describe a mathematical problem, getting easily confused and failing to resolve questions that a second grader can handle without issues.
💡 Analogy: Imagine you’re telling a story to a friend, and halfway through, you start to embellish details because you’re not sure of the exact facts. You make the story more interesting or fill in the gaps with your best guess, even if it’s not entirely accurate. That’s similar to how a model might hallucinate—it’s trying to generate content that makes sense and fits the context, even if it has to "fill in the blanks" with made-up information.
Why Do Hallucinations Happen?
Hallucinations happen because the model is designed to predict the next word or phrase based on the patterns it has learned during training. However, the model doesn’t have a built-in understanding of what is factually correct or what actually exists in the real world. It’s simply following the patterns it has learned, which means that if it sees a certain combination of words frequently, it might generate something similar—even if it’s not true.
It’s kind of like when your waiter tells you “Enjoy your meal!” and you brainlessly respond “You too!”.
In other cases, it’s even more essentially related to the architecture of LLMs: these are language models, not concept models, so when you’re breaking the data into chunks (tokens) to process it, the relationship between concepts gets destroyed.
💡 Analogy: Think of it like a well-meaning friend who always tries to finish your sentences. Sometimes they get it right, but other times they might confidently say something that’s completely off the mark, just because it "sounded right" to them. The model is doing something similar: it’s trying to predict the best possible continuation of the text, but it doesn’t always have the real-world context to ensure accuracy.
Why Are Hallucinations So Hard to Eradicate?
Even the best, most modern models hallucinate. In fact, someone is keeping a leaderboard of the estimated % of hallucinations of the most popular models. Some of the most advanced models like GPT-4o still hallucinate about 1.5% of the time (according to the leaderboard and as of 10/28/2024). Doesn’t look like a lot, but for critical applications, it’s a big deal. Would you trust a surgeon who 1.5% of the time completely botches a heart surgery, or a lawyer who 1.5% of the time completely makes up your case? (Well…maybe that last example is not unheard of…but I digress) .
Eradicating hallucinations is difficult because the model doesn’t inherently know the difference between truth and fiction -it only knows what it has learned from the vast amount of data it was trained on. Since the model is designed to be creative and flexible in generating text, it sometimes produces content that "fits" the context but isn’t grounded in reality.
Furthermore, the very architecture that makes GPT models so powerful—like the attention mechanism and the vast array of patterns it can recognize—also contributes to the risk of hallucination. The model’s goal is to generate coherent and contextually appropriate text, not necessarily to verify facts. So, hallucinations are a feature, more than a bug.
💡 Analogy: Imagine trying to teach someone to tell stories without ever teaching them what’s real and what’s fictional. They might become excellent storytellers, weaving together narratives that are engaging and logical, but without grounding, they could easily mix up facts with fiction. Similarly, even with adjustments and training, the model’s primary focus on generating text rather than verifying facts makes it hard to completely eliminate hallucinations.
So, what do we do about AI hallucinations?
While hallucinations are a known issue, there are ways to reduce their impact. Developers continuously refine the training process, adjust the model’s parameters, incorporate input/output guardrails, and utilize more accurate datasets to reduce the likelihood of hallucinations.
You may have heard about Retrieval-Augmented Generation (RAG), where a reliable information source is used to reduce the chances that the response is completely off-topic or knowledge graphs that are used to create a network of entities and relationships that can more accurately represent complex relations between concepts in a way that is “readable” by AI so that a Knowledge Graph -enhanced LLM may be able to better ground its responses.
However, because the model’s job is to predict and generate text, it’s an ongoing challenge to balance creativity with factual accuracy.
You can also contribute to limiting hallucinations by writing better prompts (e.g. being more specific, providing examples of the expected output, etc.) and especially by checking the output using your human brain. Remember: you’re supposed to use AI as your tool, not the other way around.
💡 Final Thought: Hallucinations are like a double-edged sword in AI. On one side, they represent the model’s incredible ability to generate text that flows naturally and fits the context. On the other side, they remind us that AI models, no matter how advanced, still lack true understanding and factual grounding, which means they sometimes take creative liberties that lead to inaccuracies.