There are more case filings with AI hallucinations than ever. Here’s my take on what’s causing these AI hallucinations and how to fix it*.

At Relativity Fest last week, I was honored to be part of a terrific case law panel with Kelly Twigger, Professor Bill Hamilton, US Magistrate Judge Allison Goddard and our moderator (as always) David Horrigan (who was moderating his umpteenth panel by that point).

The topic that we discussed the most (led by Professor Hamilton) was the growing number of case filings with AI hallucinations we’re seeing, which is up to 439(!) total cases, according to Damien Charlotin’s site that is tracking AI hallucination cases.

The Deterrent Value of Sanctions

Bill discussed two cases with severe sanctions – Johnson v. Dunn and ByoPlanet International v. Johansson and Gilstrap. Here are the types of sanctions issued in these two cases:

• Johnson v. Dunn: Public reprimand, disqualification from the case, and referral to the Bar.
• ByoPlanet International v. Johansson and Gilstrap: Cases dismissed without prejudice, attorney ordered to pay defendants’ attorney fees, referred to Florida Bar.

The lawyers in the ByoPlanet case kept submitting filings with AI hallucinations even after being put on notice, so they may have deserved the severe sanctions they received. As for Johnson v. Dunn, Bill ably provided a discussion of how that one happened, which sounded like more of a communication issue than an egregious instance (for which the attorneys took responsibility and apologized). In that case, the Court stated that it “assigns primary value to the deterrent function of a sanction”.

So, how “deterrent” have the sanctions been in these two cases with severe sanctions applied? Since the Johnson ruling on July 23, there have “only” been 175 cases with AI hallucinations in filings since (195 since ByoPlanet on July 15). Wow, those sanctions are really making a difference! 😉

Clearly, sanctions – even severe sanctions – are not providing enough of a deterrent to reduce (much less eliminate) case filings with AI hallucinations.

The Problem is Bigger than Lawyers

Why aren’t sanctions working? Because this problem isn’t a lawyer problem – it’s much bigger than lawyer ethics. And, no, I don’t mean pro se parties, judges(!) and experts (even AI experts are doing it). It’s a societal problem and case filings with AI hallucinations are just a single manifestation of this problem.

The problem is a phenomenon known as “automation bias”, which is the tendency for people to believe a result just because it came from a computer program or algorithm. One example of automation bias is “Death by GPS” where people drive their cars into a lake or off an unfinished bridge, just because the GPS told them to go that way (or they put their Tesla in “autopilot” mode and went to sleep). In fact, the National Highway Traffic Safety Administration (NHTSA) estimates that GPS causes over 200,000 car accidents every year in the US.

It has happened in aviation and defense as well:

• The crew of Korean Air Lines Flight 007 in 1983 was shot down over Soviet airspace because the pilots relied on an automated navigation system that had been incorrectly set up.
• Air France Flight 447 (2009) crashed when the aircraft’s autopilot disengaged during a flight through a storm when ice crystals blocked the airspeed sensors and the pilots didn’t have manual flying experience in such conditions to adjust.
• In 2003, US Army missile operators shot down a friendly British aircraft after the Patriot missile system wrongly identified it as an enemy missile.

There’s a similar site to the AI hallucinations site (which I covered here), which is tracking suspected undeclared AI usage in academic literature – they’re up to 772 reported occurrences. And, these aren’t subtle indicators of the use of AI like use of the word “crucial” or overreliance on em dashes, they are blatant phrases left in the articles like: “Certainly, here are”, “As of our knowledge cutoff”, “As an AI language model, I must”, and “new developments may have occurred since my last update”. People aren’t even proofreading their articles well enough to catch and remove those phrases.

As I noted here, two of the reasons automation bias occurs are:

• Cognitive Miser Hypothesis: People tend to prefer the path of least cognitive effort, making them more likely to accept the system’s recommendations without further thought.
• High Workload and Time Pressure: In situations with high workload or time pressure, individuals may be more likely to rely on automation to reduce their cognitive burden.

Bingo! This is what’s causing these AI hallucinations – it’s WAY bigger than lawyers or the legal system. It’s inherent in all of us, like a disease.

Training and Education is the Only Way to Fix the Problem

In that post, I noted that we’re trying to educate users about the limitations of automated systems and the potential for automation bias. Evidently, we need to try harder.

There are plenty of examples where training and education reduce problems significantly. For example, training on how to handle hazardous materials or operate heavy machinery can prevent accidents and create a safer environment. Sure, some accidents still occur, but a lot less occur than if we didn’t have that training.

We need to treat gen AI models and other forms of automation with a similar level of caution we apply to handling hazardous materials or operating heavy machinery. Training and education must start in the earliest levels – in grade school, college and law school – to be effective. It needs to be reinforced as part of any new employee training program, with regular follow-ups and reminders – especially for those who are beyond those school years.

What’s causing these AI hallucinations? Something inherent in all of us – the desire to reduce effort, especially when we need to get things done quickly. Would you skip steps when handling hazardous materials? No, because we’ve all been trained to be careful with them. We need to learn to be just as careful when working with gen AI models, especially public LLMs like ChatGPT that are a “jack of all trades, master of none” when it comes to things like accurate case citations. Handle with care!

So, what do you think? Do you agree that automation bias is what’s causing these AI hallucinations? Please share any comments you might have or if you’d like to know more about a particular topic.

*Or at least reduce it – eventually. 😊

Image created using Microsoft Designer, using the term “bewildered robots driving into a lake in a car while looking at GPS”.

Disclaimer: The views represented herein are exclusively the views of the author, and do not necessarily represent the views held by my employer, my partners or my clients. eDiscovery Today is made available solely for educational purposes to provide general information about general eDiscovery principles and not to provide specific legal advice applicable to any particular circumstance. eDiscovery Today should not be used as a substitute for competent legal advice from a lawyer you have retained and who has agreed to represent you.