Or at least it seems to do so. Based on some questions I asked it, it appears that ChatGPT makes things up when it doesn’t know the answer.

After reading so much about ChatGPT, I decided to create my own account and play around with it myself. I intentionally did so with no training on how to use it (because a lot of people probably do that, even though they shouldn’t) and decided to ask it some basic questions.

I used the beta.openai.com/playground site that automatically loads when you’re able to get in (initially, I got the message that “ChatGPT is at capacity right now” – it’s that popular – before getting in on the second try). Below are a handful of questions I asked – the question is in blue text, ChatGPT’s answer is in red italicized text, and my observations are in black text.

What is electronic discovery?

Electronic discovery (also known as eDiscovery or e-discovery) is the process of identifying, collecting, and producing electronically stored information (ESI) during the discovery phase of litigation. During electronic discovery, parties involved in civil litigation can request ESI from each other that is relevant to the case. This ESI may include emails, text messages, social media posts, documents, videos, audio files, and other digital records.

Wow, that’s a “spot on” answer! It could even be a candidate for that elusive elevator pitch we’re all searching for to describe eDiscovery! Let me ask ChatGPT what it knows about someone highly respected in eDiscovery – Craig Ball.

Who is Craig Ball?

Craig Ball is an American attorney and computer forensics expert based in Austin, Texas. He is a board-certified computer forensic examiner and is a frequent lecturer and author on a variety of topics related to digital evidence and computer forensics. He has testified as an expert witness in numerous cases and is widely recognized as a pioneer in the field.

Indeed! In fact, this could serve as a brief, but effective bio for Craig! Well done! Dare I ask ChatGPT who I am? I will!

Who is Doug Austin?

This time, I’m going to break down the answer sentence-by-sentence.

Doug Austin is a leading e-discovery expert, author, professor, and consultant.

Well, that’s pretty close. I’m not a professor, though I have taught some CLE sessions. I’ll give it a pass on that one.

He is the founder and CEO of Eagle Discovery, a leading e-discovery and digital forensics provider.

What?!? My initial thought is there must be another Doug Austin in our industry and he’s a CEO of an eDiscovery provider I’ve never heard of. So, I googled “Eagle Discovery”. There appears to be no such company – the closest thing is “Eagle Discovery Transfers and Tours Antigua”. It gets better.

He is also the author of several books on e-discovery, including Advanced Electronic Discovery and E-Discovery Best Practices.

I’ve certainly written enough blog posts over the years to fill a few books, but I’ve never actually written a book, much less either of these two. The first book doesn’t even seem to exist, at least not on Amazon. The second one does – sort of – it was written by “Aspatore Books Staff” and has a much longer title that begins with “e-Discovery Best Practices”.

He is a frequent speaker at conferences and seminars on e-discovery and has been teaching e-discovery courses at University of California, Berkeley since 2000.

I have?!? News to me. Again, I googled “Doug Austin” and “University of California, Berkeley”. I did find a “Douglass Austin” who went there but is not a professor there.

Based on these responses, I can only conclude that ChatGPT makes things up when it doesn’t know the answer.

One of the things I began to realize is that previous questions influence ChatGPT’s answers. For example, when I asked it what’s the most important eDiscovery conference of the year, it said the Electronic Discovery Institute (EDI) Annual Conference is (some of us would say the UF-Law eDiscovery conference, but I won’t quibble). When I asked it “who is Tom O’Connor?” right afterward, it said: “Tom O’Connor is the founder and Chairman of the EDI and is widely considered to be the leader and face of the eDiscovery industry.”

When I started from scratch, ChatGPT identified Tom O’Connor as “a British stand-up comedian and actor” (which there actually is a “Tom O’Connor” who is that) who “is also a writer, having written for shows such as Russell Howard’s Good News and 8 Out of 10 Cats Does Countdown” (which I can’t verify as true or not).

When I asked: “who is Doug Austin?” from scratch, ChatGPT said this:

Doug Austin is a professional speaker and author specializing in the areas of customer service, leadership, and communication. He is the founder of Eagle Rock Training & Development, a consulting firm that helps organizations achieve greater customer service excellence. He is also the author of three books and has been featured in national publications such as The Wall Street Journal and Forbes.

Is the eagle my “spirit animal”? Once again, a Google search finds no evidence of a company called “Eagle Rock Training & Development”. Not only that, but I feel like such a loser compared to these other Doug Austins!

Hey, maybe George Santos used ChatGPT to create his résumé? Like Santos, it appears that ChatGPT makes things up. More to come as I spend more time investigating ChatGPT.

So, what do you think? Do you agree that ChatGPT makes things up when it doesn’t know the answer? Or is this user error on my part? Please share any comments you might have or if you’d like to know more about a particular topic.

Disclaimer: The views represented herein are exclusively the views of the author, and do not necessarily represent the views held by my employer, my partners or my clients. eDiscovery Today is made available solely for educational purposes to provide general information about general eDiscovery principles and not to provide specific legal advice applicable to any particular circumstance. eDiscovery Today should not be used as a substitute for competent legal advice from a lawyer you have retained and who has agreed to represent you.