Redgrave LLP compared Relativity aiR for Review to active learning. Here are 5 lessons from their independent study of aiR for Review.

The write up was provided by Robert Keeling and Ray Mangum of Redgrave LLP in this blog post titled (wait for it!) Results Are In: 5 Lessons from an Independent Study of aiR for Review (available here). As they discussed, Redgrave conducted a head-to-head study comparing Relativity aiR for Review, a generative AI review workflow, against a traditional first-pass managed review workflow using active learning. They “chose a deliberately difficult document population involving ~45,000 documents from a real-world public data set and a nuanced responsiveness standard tied to pharmaceutical marketing, controlled substances, and federal compliance obligations.”

As the authors note: “documents were not responsive simply because they mentioned opioids, sales activity, or drug promotion. They had to contain evidence related to compliance with, violation of, or reckless disregard of federal requirements. That made the exercise less about finding documents ‘about’ a topic and more about applying judgment to the contents of the document.”

Nice test.

A subject-matter expert then conducted a blind review of a random sample to establish the ground truth against which both workflows were measured.

The full report gets into the methodology, statistical analysis, and validation process in detail. But for legal teams thinking about how generative AI may fit into real review workflows, five practical lessons stood out. Here’s one of them:

Judge speed by the quality of the result.

The efficiency difference in the study was striking. The aiR for Review workflow required approximately 18 hours of attorney time, while the active learning managed review workflow required approximately 1,123 hours from a 24-person team over seven business days.

That represents a roughly 98 percent reduction in cumulative human hours.

Time savings at that scale are hard to ignore. But speed alone is not the full story. The more important point is that aiR for Review achieved those efficiency gains while also delivering higher recall and lower elusion in this study.

In other words, this was not simply a faster way to get through documents. It was a faster workflow that found more responsive material and left fewer important documents behind. That combination is what makes the result meaningful. Faster review is valuable, but faster review that also improves completeness is far more valuable.

The business case for generative AI in review is not just about doing the same work faster. It is about helping legal teams move faster without accepting more risk, while freeing attorneys to spend more time on analysis, strategy, and advocacy.

So, what are the other 4 of 5 lessons from their independent study of aiR for Review? Find out here, it’s just one click! You can’t study their study without clicking – unless you click to get the 17-page full report here. 😉

So, what do you think? How are you evaluating generative AI review workflows? Please share any comments you might have or if you’d like to know more about a particular topic.

Image created using DALL-E, 3 using the term “robot lawyer wearing a suit looking through a microscope at a workstation”.

Disclosure: Relativity is an Educational Partner and sponsor of eDiscovery Today.

Disclaimer: The views represented herein are exclusively the views of the author, and do not necessarily represent the views held by my employer, my partners or my clients. eDiscovery Today is made available solely for educational purposes to provide general information about general eDiscovery principles and not to provide specific legal advice applicable to any particular circumstance. eDiscovery Today should not be used as a substitute for competent legal advice from a lawyer you have retained and who has agreed to represent you.