As the Court stated: “The irony”, in rejecting a court declaration from an expert on AI and misinformation that contained fake citations.

Last year, according to The Stanford Daily (more irony!), Stanford communication professor Jeff Hancock, an AI and misinformation expert, used GPT-4o to create a court declaration on deepfakes. For the declaration, Hancock surveyed scholarly literature on the risks of deepfake technology in spreading misinformation using GPT-4o, but failed to factcheck the citations it generated. Unfortunately for him, the AI agent generated two “hallucinated citations” and also introduced an error in a list of authors for an existing study.

Whoops.

In a subsequent filing he submitted to the Court in support of a motion to file an amended declaration, Hancock stated: “I did not intend to mislead the Court or counsel. I express my sincere regret for any confusion this may have caused. That said, I stand firmly behind all of the substantive points in the declaration.”

The Court was not moved. In a ruling Friday, Minnesota District Judge Laura M. Provinzino stated:

“The irony. Professor Hancock, a credentialed expert on the dangers of AI and misinformation, has fallen victim to the siren call of relying too heavily on AI—in a case that revolves around the dangers of AI, no less. Professor Hancock offers a detailed explanation of his drafting process to explain precisely how and why these AI-hallucinated citations in his declaration came to be…And he assures the Court that he stands by the substantive propositions in his declaration, even those that are supported by fake citations…But, at the end of the day, even if the errors were an innocent mistake, and even if the propositions are substantively accurate, the fact remains that Professor Hancock submitted a declaration made under penalty of perjury with fake citations.”

Continuing, she said: “The Court thus adds its voice to a growing chorus of courts around the country declaring the same message: verify AI-generated content in legal submissions!”

While noting that “Plaintiffs seem to accept that Professor Hancock is qualified to render an expert opinion on AI and deepfakes, and the Court does not dispute that conclusion”, Judge Provinzino also noted: “Nevertheless, Plaintiffs argue that the Hancock Declaration should be excluded in its entirety and that the Court should not consider an amended declaration.”

To which she responded: “The Court agrees. Professor Hancock’s citation to fake, AI-generated sources in his declaration—even with his helpful, thorough, and plausible explanation…—shatters his credibility with this Court. At a minimum, expert testimony is supposed to be reliable…More fundamentally, signing a declaration under penalty of perjury is not a mere formality; rather, it ‘alert[s] declarants to the gravity of their undertaking and thereby have a meaningful effect on truth-telling and reliability.’… Moreover, citing to fake sources imposes many harms, including ‘wasting the opposing party’s time and money, the Court’s time and resources, and reputational harms to the legal system (to name a few).’”

In ruling on the motion to amend the declaration, Judge Provinzino stated: “To be sure, the Court does not believe that Professor Hancock intentionally cited to fake sources, and the Court commends Professor Hancock and Attorney General Ellison for promptly conceding and addressing the errors in the Hancock Declaration. But the Court cannot accept false statements—innocent or not—in an expert’s declaration submitted under penalty of perjury. Accordingly, given that the Hancock Declaration’s errors undermine its competence and credibility, the Court will exclude consideration of Professor Hancock’s expert testimony in deciding Plaintiffs’ preliminary-injunction motion.”

When an expert on AI and misinformation submits a declaration generated by AI containing misinformation, that’s not just irony – it’s the ultimate irony. It’s not just lawyers that need to fact check their work; it’s everybody. Hancock should know that better than anybody.

Hat tip to Judge Andrew Peck for the heads up on the court ruling!

So, what do you think? Are you surprised that an expert on AI and misinformation submitted a declaration that contained fake citations? Please share any comments you might have or if you’d like to know more about a particular topic.

Ruling link courtesy of Thomson Reuters.

Disclaimer: The views represented herein are exclusively the views of the authors and speakers themselves, and do not necessarily represent the views held by my employer, my partners or my clients. eDiscovery Today is made available solely for educational purposes to provide general information about general eDiscovery principles and not to provide specific legal advice applicable to any particular circumstance. eDiscovery Today should not be used as a substitute for competent legal advice from a lawyer you have retained and who has agreed to represent you.