The statement “more accurate (Gen)AI doesn’t always reduce total document review costs” may be shocking. But Jeremy Pickens of Elevate backs up that statement with several cost comparisons.
The whitepaper titled Why More Accurate (Gen)AI Doesn’t Always Reduce Total Document Review Costs, available here for download, is – as Jeremy (aka “Dr. J” to many of us) – “Comfortably Numb-ers Driven” (see what he did there? 😉). As the intro page notes:
Most discussions about AI-powered document review focus on accuracy (precision and recall). The assumption is straightforward: if a model is more accurate, review volumes fall, and costs follow. Reality is more complicated. A more accurate model can be more expensive than a less accurate one, and that expense can outweigh savings from decreased review. In some matters, the approach that reviews the fewest documents is not the approach that delivers the greatest savings.
The reason is that document review costs are shaped by more than simply number of documents reviewed. Technology costs, subject-matter expert effort, corpus size, and richness all interact with review volume in ways that can fundamentally change the economics of a matter. This complexity creates a challenge for law departments and law firms evaluating AI-powered workflows. How should savings be measured? Which costs matter most? And how do you determine whether a new approach is genuinely better than the one already in use?
In this whitepaper, Jeremy (who just earned individual recognition in the Chambers and Partners Litigation Support Guide 2026)presents a practical framework for answering those questions. Using quantitative examples drawn from real-world eDiscovery economics, he demonstrates why precision and recall tell only a part of the story, and why the most economical approach often depends on the shape of the matter itself.
That’s what the intro page says. Without stealing too much of Jeremy’s thunder, here are some notable observations from the whitepaper:
- The motivation for this article came from a recent law firm report extolling the virtues of GenAI because it had saved 50% of the cost relative to linear review. But that’s really not the next best alternative, as linear review “is a dreadfully weak baseline that is trivial to beat and therefore does not convey the truest picture.”
- In the law firm report, the collection was only about 3% rich. An older supervised machine learning TAR 1 approach, which may have yielded 10% precision at 90% recall, would have saved almost 70% over the cost of linear review – even more than the GenAI approach.
- From a per-document technology cost standpoint, Jeremy states that the current standard industry pricing for traditional forms of TAR (both 1 and 2) is $0/document. GenAI, by contrast, incurs a cost for every document analyzed. For purposes of the sensitivity analysis, Jeremy assumes $0.10 per document, though he acknowledges pricing varies – could be higher or lower.
- TAR and GenAI approaches both require subject matter expert (SME) time to either train the model for TAR or prompt engineering for GenAI. For the sake of simplicity, Jeremy chose to assume a fixed 20 hours of SME time for both approaches.
- Document richness (the percentage of responsive documents within the corpus) has a significant impact on the analysis. Lower richness means “the entire review-cost component shrinks for all AI/machine learning methods, which mutes precision’s advantage and lets cheaper-to-run methods catch up.” So, improvements in precision for AI methods provide the greatest return on matters containing a larger percentage (i.e., higher richness) of responsive documents.
A major portion of the paper is dedicated to six comparisons that look at various scenarios (20% richness vs. 5%, and matter size from 10,000 to 100,000 to 1 million to 5 million documents) for linear review, supervised TAR 1 and TAR 2 and GenAI. Jeremy explains the parameters clearly so that you can not only understand the six comparisons, but also what it might look like if you change any of the assumptions.
As Jeremy and Elevate note in the Conclusion: “in the world currently being painted by most vendor marketing departments, GenAI-based TAR 1 is the only way forward, and nothing else matters…No approach is universally best across all matters. The most effective document review strategies are not driven by technology preferences, but by data, economics, and expertise.”
That’s why more accurate (Gen)AI doesn’t always reduce total document review costs. “Your eDiscovery pain will recede when you are comfortably numbers-driven.”
Again, the paper is available for download here.
So, what do you think? Is your organization conducting this type of analysis for your review projects? Please share any comments you might have or if you’d like to know more about a particular topic.
Image created using DALL-E 3, using the term “robot lawyer listening to rock music while looking at spreadsheets”.
Disclaimer: The views represented herein are exclusively the views of the author, and do not necessarily represent the views held by my employer, my partners or my clients. eDiscovery Today is made available solely for educational purposes to provide general information about general eDiscovery principles and not to provide specific legal advice applicable to any particular circumstance. eDiscovery Today should not be used as a substitute for competent legal advice from a lawyer you have retained and who has agreed to represent you.