Can you spot a deepfake? Based on the article I covered earlier today, I decided to see how well I could do in spotting deepfakes.

The article I’m referring to is Judicial Approaches to Acknowledged and Unacknowledged AI-Generated Evidence (and available here), which is the latest article authored by Maura R. Grossman and Hon. Paul W. Grimm (ret.), which discusses a variety of considerations for deepfakes, including potential changes to rules of evidence and practical suggestions for “what lawyers and courts might do to deal with acknowledged and unacknowledged AI-generated evidence now, since any rules change is likely years away”.

One of the overriding themes of the article is that people aren’t very good at determining the authenticity of media – i.e., we’re not good at differentiating real media from deepfakes. The article mentions one online tool developed by MIT Media Labs and maintained by Northwestern University called “Detect Fakes”, where you’re presented with a series of images and must decide if they’re real or not. So, I decided to try this tool (and a few others) to see how well I could do in spotting deepfakes.

Detect Fakes

This site is limited to images and the scoring system appears to be a bit wonky. It displays an image and asks you to vote on whether the image is real or fake. Once you vote, it asks you again to “Take a second look” and vote again. Then (in theory), it gives you a point for each correct answer, so if you chose correctly both times on an image, you should get 2 points. What I noticed is that it doesn’t always give you both points, even if you get both right, and it sometimes gives you a point, even if you get both wrong. So, you may want to keep your own scoring. Also, it doesn’t explain WHY an image is real or fake, which is annoying.

On this site, I correctly identified 30 out of 50 images, which (since I made the same choice both times for each image) was essentially 15 out of 25 correct answers for unique images. The site pointed out that the “average participant identified 38 correctly at this point” (or 19 out of 25 unique images). So, I got about 60% of the answers correct, which was below average. Very few of the images were obvious.

Spot the Deepfake

This is a ten-question quiz, which asks general questions about deepfakes and also asks you to differentiate real images and videos from deepfakes in about half the questions. I got seven out of ten correct on this one, missing one image and one video. Unlike the “Detect Fakes” site, this site discusses the answers, including why an image or a video is real or a deepfake.

Quiz: could you spot a deepfake?

This is a simple five-question quiz, asking you to review five videos and determine whether each is real or a deepfake. I did badly at this one – only one out of five!

There are a few other deepfake quiz sites out there – offered by companies who want to sell you their products, so I’m not mentioning them here. They’re easily found in a Google search.

Can you spot a deepfake? Apparently, I can’t do so reliably. Overall, I got 23 out of 40 (including 5 questions in the middle quiz, which were questions about deepfakes, not asking me to identify them). Several of my correct and incorrect answers were guesses based on what I thought were indicators of real or fake – for the wrong answers, they were obviously not the correct indicators. 🤪

Grossman and Grimm’s article discusses that there are two basic types of deepfake detection: (1) inference-based (looking for signals in the actual media content that do not sync, line up, or match properly), and (2) provenance-based (which involves the review of metadata, such as timestamps and GPS coordinates, to look for signals suggesting the use of AI in generating or manipulating the media). What I did – or attempted to do – was inference-based. Experts can probably do it better, but nothing is foolproof. That’s the challenge.

So, what do you think? Can you spot a deepfake – at least better than I can? 😉 Please share any comments you might have or if you’d like to know more about a particular topic.

Image created using GPT-4’s Image Creator Powered by DALL-E, using the term “robot sitting at a desk in front of a computer showing a picture of another robot”.

Disclaimer: The views represented herein are exclusively the views of the authors and speakers themselves, and do not necessarily represent the views held by my employer, my partners or my clients. eDiscovery Today is made available solely for educational purposes to provide general information about general eDiscovery principles and not to provide specific legal advice applicable to any particular circumstance. eDiscovery Today should not be used as a substitute for competent legal advice from a lawyer you have retained and who has agreed to represent you.