How should judges manage the growing issue of deepfakes in court? A new paper seeks to provide guidance and suggest changes.

The paper titled Deepfakes in Court: How Judges Can Proactively Manage Alleged AI-Generated Material in National Security Cases (and available here) was authored by eight different authors, including Maura R. Grossman, Hon. Paul W. Grimm (ret.), five authors from Northwestern University, and Minnesota District Judge John Tunheim. It addresses the growing challenges courts face due to the rise of deepfake and AI-generated materials (AIM). The authors focus on managing deepfake evidence, particularly in high-stakes national security cases and elections.

In the Abstract of the paper, the authors lay out the problem of deepfakes in court in plain language, as follows:

“We must confront two possibilities: first, that evidence presented is AI generated and not real and, second, that other evidence is genuine but alleged to be fabricated. Technologies designed to detect AI generated content have proven to be unreliable, and also biased. Humans have also proven to be poor judges of whether a digital artifact is real or fake. There is no foolproof way today to classify text, audio, video, or images as authentic or AI generated, especially as adversaries continually evolve their deepfake generation methodology to evade detection. Thus, the generation and detection of fake evidence will continue to be a cat and mouse game. These are not challenges of a far-off future, they are already here. Judges will increasingly need to establish best practices to deal with a potential deluge of evidentiary issues.”

One of the notable areas of discussion is how the current state of deepfake detection technologies is that they are inconsistent and unreliable. The authors collected 100 real videos, as well as 100 well-known deepfakes and they generated their own deepfakes. They then tested four well-known deepfake detectors, none of which scored very well. Many of the deepfakes which would have been easily detected by a human were missed by the deepfake detectors, which were biased toward labeling videos as real, so their ability to detect the deepfakes “do not provide confidence”, according to the analysis.

Hoping for watermarks or cryptographic signatures embedded in AI-generated content to be the answer? Alas, they are far from foolproof. Skilled actors can easily strip or alter watermarks and cryptographic signatures from AIM and malicious actors can avoid major platforms and use freely available tools to generate AIM that is untraceable and lacks any identifying marks.

The paper also provides an “election-interference hypothetical” where a 2028 U.S. presidential election where a candidate alleges her opponent is using deepfakes to spread disinformation, intimidate voters, and sway public opinion. The deepfakes show the candidate in compromising situations, which impact voter turnout and support, and could also lead to claims under the voting rights act and for defamation. That couldn’t happen today, right? 😉 It could – and already has – as the authors point out with examples both in Russia last year and the US earlier this year. Sigh.

The paper also devotes a section to discussing the Federal Rules of Evidence (FRE) framework, where courts traditionally admit evidence that meets low bars for relevance and authenticity, where the proponent only needs to show that the evidence is more likely than not what it claims to be, under FRE rules 401 (Relevance) & 901 (Authenticity).

How should a court address these challenges? With the GPTJudge Framework for addressing allegedly fake AIM evidence developed by Maura R. Grossman, Hon. Paul W. Grimm (ret.), Daniel G. Brown, and Molly (Ximing) Xu in their article published last year (and covered by me here): The GPTJudge: Justice in a Generative AI World – a structured approach proposed by the authors for courts to manage and resolve disputes over the authenticity of alleged AIM in legal proceedings. Key aspects of the GPTJudge Framework include:

• Judges are encouraged to take a proactive approach by requiring parties to raise issues regarding AIM early in the litigation process, such as during pre-trial conferences (under the Federal Rules of Civil Procedure 16 and 26(f)).
• The importance of holding pre-trial evidentiary hearings to address AIM admissibility. Judges should evaluate whether the evidence meets the authenticity and relevance requirements under the Federal Rules of Evidence, such as Rules 401, 403, and 901.
• Judges should not merely assess AIM evidence based on a simple “more likely than not” authenticity standard (as traditionally done under Rule 901) but should balance the risks associated with admitting potentially fake evidence, considering the negative consequences if the evidence is later found to be fake.

One thing I love is that the framework encourages allowing discovery to corroborate or refute allegations that certain evidence is AIM. Both sides can use expert testimony to support or challenge the authenticity of evidence.

The last section discusses growing calls to amend the Federal Rules of Evidence, including modifications that Grossman and Grimm have proposed to Rule 901 for possible deepfakes (previously covered by me here and here), as well as other proposed changes.

In the Conclusion, the authors note: “Given the ease with which anyone can create a convincing deepfake, courts should expect to see a flood of cases in which the parties allege that evidence is not real, but AI generated. Election interference is one example of a national security scenario in which deepfakes have important consequences. There is unlikely to be a technical solution to the deepfake problem.” So, it’s up to the judges to “proactively address disputes regarding alleged deepfakes, including through scheduling conferences, permitted discovery, and hearings to develop the factual and legal issues to resolve these disputes well before trial.”

The 52-page paper covers a lot of ground and gets “deep” into the issue of deepfakes in court (see what I did there?). 😉 Regardless, these are the issues we will all be wrestling with ASAP – if we’re not already. You can download the paper here.

So, what do you think? Are you concerned about the potential of deepfakes in court? Please share any comments you might have or if you’d like to know more about a particular topic.

Image created using GPT-4’s Image Creator Powered by DALL-E, using the term “robot sitting at a desk in front of a computer showing a picture of another robot”.

Disclaimer: The views represented herein are exclusively the views of the authors and speakers themselves, and do not necessarily represent the views held by my employer, my partners or my clients. eDiscovery Today is made available solely for educational purposes to provide general information about general eDiscovery principles and not to provide specific legal advice applicable to any particular circumstance. eDiscovery Today should not be used as a substitute for competent legal advice from a lawyer you have retained and who has agreed to represent you.