In preparing for two webinars in the next two weeks on proportionality in discovery, I thought I would take a fresh look at the number of proportionality disputes and conduct a search on how many proportionality disputes there were in eDiscovery case law for 2021. The result is that proportionality disputes continue to rise – dramatically.
My go-to source for eDiscovery case law is, of course, eDiscovery Assistant and I added 2021 numbers to the four years I looked at last year (2017-2020). So, here are the number of case law rulings that included proportionality disputes over the last five years (or just look at the graphic above):
- 2017: 313 case law rulings
- 2018: 403 case law rulings (up 28.8%)
- 2019: 473 case law rulings (up 17.4% over the previous year, up 51.1% since the first year)
- 2020: 868 case law rulings (up 83.5% over the previous year, up 177.3% since the first year)
- 2021: 1,206 case law rulings (up 38.9% over the previous year, up 285.3% since the first year)
And the 2021 numbers may still rise more! Sometimes, eDiscovery Assistant is loading in straggler cases from the prior year for two or three months into the next year.
Here are the yearly totals of case law rulings with proportionality disputes represented graphically:
Those are pretty telling statistics on why there’s a need for a New Framework on discovery proportionality!
So, just a reminder that I will be moderating a webinar today conducted by the George Washington University Law School (GWU) at 1pm ET (details on how to register here) and I will be moderating another webinar conducted by ACEDS on Wednesday, February 23 at 1pm ET (details on how to register here). Both will be discussing the public comment version of the New Framework on discovery proportionality published by GWU last month. Hope to see you there!
So, what do you think? Are you surprised how much the number of case law rulings with proportionality disputes has risen? Please share any comments you might have or if you’d like to know more about a particular topic.
Disclaimer: The views represented herein are exclusively the views of the author, and do not necessarily represent the views held by my employer, my partners or my clients. eDiscovery Today is made available solely for educational purposes to provide general information about general eDiscovery principles and not to provide specific legal advice applicable to any particular circumstance. eDiscovery Today should not be used as a substitute for competent legal advice from a lawyer you have retained and who has agreed to represent you.
I’ve got two major comments. About issues that I think seriously affect any attempt at having a proportionality discussion.
(1) The first issue is that the stages are not independent. While I appreciate the value in trying to lay out a cost calculator by breaking down the cost of the various components, in reality the components themselves do not have a fixed price. Their price is affected by what happens in the other components. I could write a 20 page treatise on this and all the ways that different stages affect each other, but for a few quick examples, imagine the first pass review fee. That hourly rate might be very cheap, but if what you’re buying is low quality reviewers, then more money will have to be spent in second pass review, because you might have to do more QC. So $5/hour less in first pass might translate to more hours in the second pass for an overall cost that is higher even if one individual stage is cheaper. The second example is hosting+TAR cost. The whole point of TAR is to save review costs. And suppose you go with the cheapest hosting+TAR price.. but as a result get an underperforming TAR engine, and thus have to spend more on review to get to an acceptable stopping point. Then the lower cost in one stage does not actually lower your overall cost, because it makes a different stage more expensive.
I would be very interested to know how this kind of thing is addressed in the new proportionality framework.
(2) The second issue is that, even if the stages were completely independent, how could any discussion of proportionality even happen at all if you cannot get a general sense of what your (hopefully TAR-based) review cost is going to be? That is, review is by far the single biggest component of the overall cost, correct? The RAND study a decade ago placed it at 70% of the pie, but I think it’s even higher today (80%? 90%?) because other tech-related prices such as GB hosting have dropped dramatically. So review makes up an even higher proportion of.. well.. proportionality. Which means that you need to be able to estimate how much review you’re going to do, right?
So this brings me to the fact that for over a decade I have been making offers to people to do TAR simulations, to show them and give them a sense of how much review they would actually have to do, on their own real data, if they followed different kind of protocols, i.e. did their review in different ways. The vast majority of this industry shies away from this, completely. Once a matter is done, it’s done, and they never want to look back on how else it could have been done, and how the costs would change under various ways of doing things. In other words, despite the literal wording of Principle 6 about how the producing party is in the best position to evaluate the (costs of) tech, almost no one ever does that evaluation. No one works to give themselves a sense of what the tech might do in various situations and circumstances. Of how this or that factor might cause the tech to perform.
And since almost no one knows how the tech will perform in this or that circumstance, almost no one in the industry is prepared to have a real discussion on proportionality. Why? To summarize, it’s because (a) review is the biggest pie slice in the proportionality discussion, (b) review costs are variable and tech/matter dependent, and (c) almost no one fulfills their Principle 6 obligation to do evaluations to understand that variability and dependency.
In my opinion, any discussion of proportionality MUST start with a discussion of evaluation of any factor that affects review costs.. the largest of which is the use (and non-use, for that matter, too!) of TAR.
Thanks, Dr. J. — terrific insights as usual. We need this kind of dialogue from everybody involved in the discovery process — lawyers, judges, and certainly the practitioners conducting discovery — to make the Framework a success. I think the cost calculator in particular needs a lot of work and feedback from people like you who understand the process — in my opinion, it’s a work in progress.
I will note that the project team is working on a TAR estimation model that they are planning to roll out in time for the next webinar on March 10 (yes, smack dab in the middle of Legalweek, unfortunately). It will be great to have feedback from people like you on that, as I’m sure it will need considerable scrutiny as well.