Eegads! A Saturday morning post* from eDiscovery Today? Recently, I’ve started writing posts for EDRM for them to publish on their blog. But, of course, the reader of eDiscovery Today are my first priority (and my bread and butter), so I want to make sure you can enjoy that content as well. So, ICYMI, here are 3 reasons people avoid using TAR in their cases and 4 ways to promote using it.
Recently, I received an email from a reader of eDiscovery Today in which she asked: Why don’t people use Technology Assisted Review (TAR)/predictive coding (and related technologies) more?
It’s a great question and the use of predictive coding (or lack thereof) is illustrated in the State of the Industry report that eDiscovery Today publishes every year, which is sponsored by EDRM. In this year’s report (which is summarized here), out of 281 respondents, only 25.9% of respondents said they use predictive coding in all or most of their cases, while 36.3% of respondents said they use it in very few or none of their cases.
Even worse, those numbers reflect less usage than the previous year, where 31.1% of respondents said they use predictive coding in all or most of their cases (5.2% higher than this year), and 32.8% of respondents said they use it in very few or none of their cases (3.5% lower than this year). In other words, a smaller percentage of respondents uses it most of the time and a larger percentage of respondents hardly uses it.
Does this mean the use of TAR is regressing? Not necessarily. I think the fact that this year’s survey had over 50% more respondents than last year’s survey (which had 183 respondents) means that it illustrates a broader, more realistic view of the industry’s use of it. Keep in mind these are still people who chose to take an eDiscovery survey, so those results are probably still more optimistic than the legal community overall.
So, why don’t more people use TAR? Here are 3 reasons:
Lack of Understanding: Simply put, people avoid what they don’t understand. And – despite many efforts to educate the legal community on it – they still don’t understand how machine learning technologies work. To many, it’s a “black box” that they don’t understand and feel ill equipped to defend in court if they are required to do so.
Conversely, most legal professionals think they understand keyword search (because they learned it in law school with Westlaw or Lexis, or they know how to perform a Google search), but many don’t understand keyword search best practices when it comes to discovery, including testing search results and the null sets for those results. This recent example illustrates that many legal professionals don’t understand keyword search either. But most think they understand it, so they prefer that approach over something they know they don’t understand.
Unwillingness to Change: Even if they do understand the technology and recognize the efficiency and effectiveness of TAR, many legal professionals are still unwilling to change their workflows to incorporate it. When there are a lot of documents to review, the mindset is to put more reviewers on it. In some cases, this may be because there is a concern that offering a different approach may reduce billable hours (there, I said it!), while in other cases, it may simply be a resistance to change.
This case that we discussed in the April case law webinar illustrates that – the defendants were looking at reviewing 225,000 documents at a cost of between $140,000 and $235,000, with a team of 15 reviewers. Yet, there was no mention of the consideration of TAR anywhere. It’s natural for people to stick with what they know unless forced to change, even if they recognize there may be a more efficient way to do it.
More Transparency is Expected with TAR: Another reason that legal professionals avoid using TAR is that opposing parties tend to want more transparency on TAR processes than they do on the traditional keyword search approach. Why? That’s a good question and it probably relates to reasons 1 and 2 above.
Maura R. Grossman and Gordon V. Cormack noted in their paper last year The eDiscovery Medicine Show (which I covered here) the “misconception…that only the TAR tool should be subject to validation, while keyword culling and manual review should be exempt, as they have always been.” Legal professionals tend to assume that their opposing counterparts know what they’re doing when it comes to keyword search (even though many don’t), but they only tend to want proof that they know what they’re doing when they’re using TAR.
In last year’s State of the Industry report, Judge Andrew Peck (ret.), who authored the first ruling to approve the use of TAR over ten years ago in Da Silva Moore, stated as a reason that more people don’t use TAR: “Part of the problem remains requesting parties that seek such extensive involvement in the process and overly complex verification that responding parties are discouraged from using TAR.”
It shouldn’t be that way, but it is. So, how do we get people to start using TAR? Here are 4 ways to promote using TAR in more cases.
Two things to note before I begin:
- Promoting the use of TAR is not just the responsibility of just one group here. Making a real difference involves action from lawyers, courts and the software providers that offer TAR solutions; and
- These are not just my thoughts – they are ideas and recommendations I’ve heard from other experts in the industry over the years.
What Lawyers Can Do
Of course, lawyers are the key to increased TAR use in litigation. But, as I mentioned last time, there’s a: 1) lack of understanding of TAR, 2) an unwillingness to change what they’re doing, and 3) a desire to avoid having to be more transparent about their eDiscovery process when using TAR. Let’s talk about two ways to address these concerns:
Test TAR on a Prior Case First: It’s understandable that lawyers are reluctant to use TAR on a case that is active with tight deadlines – that’s not the best time to experiment and learn new workflows. Practicing the use of TAR with a case that’s already completed enables you to get used to the process without any deadlines or required outcomes to worry about, while giving you an outcome that has already occurred to measure against. If you’re missing important documents that you found in the case before, now is the time to adjust and learn how to identify them more accurately. Practicing TAR on a prior case may also uncover important documents you missed the first time, showing you just how powerful TAR can be!
Use TAR on Documents Produced to You: The biggest argument that lawyers may have regarding testing TAR on a prior case is the “ain’t nobody got time for that” argument. While you should make time to test TAR before actively using it on live cases, I get it – lawyers are busy and that’s not always feasible. What you can do is use TAR to conduct a review of documents that were produced to you. You have to review them anyway, right? Why not let the technology help you find the documents that are important?
One of the biggest advantages of testing TAR before using it on an active case and/or using TAR on documents produced to you is eliminating the defensibility requirement. You don’t have to be transparent when using it for your own needs and you can work out the process before using it for review and production in a case where you may have to be transparent.
What Courts Can Do
Speaking of transparency, let’s discuss what the courts can do to help promote the use of TAR. As Maura R. Grossman and Gordon V. Cormack noted in their paper The eDiscovery Medicine Show, “The Ball is in The Courts’ Court”. Maura and Gordon, while acknowledging Sedona Principle 6, stated “producing parties should show—and the courts should demand that they show—the reasonableness of their eDiscovery search and review processes, as well as the resulting production, by hewing closely to tools, methods, and procedures that have been scientifically vetted and shown to be valid and reliable.”
Not that every case requires (or should require) that level of defensibility. Lacking a reason to otherwise question them, Sedona Principle 6, which states that “[r]esponding parties are best situated to evaluate the procedures, methodologies, and technologies appropriate for preserving and producing their own electronically stored information”, should be a sufficient barometer for parties’ handling of discovery in many cases without disputes over potential deficiencies in a party’s discovery process. Heck, even Judge Andrew Peck (ret.), the author of the first ruling to approve the use of TAR (and a strong advocate of TAR approaches), invoked that principle in Hyles v. New York City in ruling the defendant didn’t have to use TAR because of Sedona Principle 6.
What courts can do, most of all is:
Be Consistent: Don’t hold TAR based approaches to a different standard than approaches based on keyword search. What’s good for the goose is good for the gander! You can quote me on that! 😉 If TAR based approaches should be transparent, so should keyword search-based approaches.
Lawyers can help here as well, by (at least) demanding the same level of transparency in TAR and keyword search. Too many legal professionals get keyword search wrong to assume people know what they’re doing and give them a pass.
What Software Providers Can Do
Let’s not leave software providers out of the mix here. They need to:
Make TAR (and Other Machine Learning Processes) More Intuitive: Part of the reason that lawyers eschew TAR is that software platforms make the workflow so different than what they’re used to doing with a traditional keyword search and review approach.
To the extent possible, machine learning needs to be baked into existing workflows so that lawyers almost don’t even realize they’re using TAR. All lawyers use machine learning technology these days – in platforms ranging from Netflix to Pandora to Amazon (and many more). It’s intuitive for them there, why can’t it be in eDiscovery platforms too? The good news is that several platforms seem to be moving in that direction, so this way of promoting TAR is already well underway.
Making strides in the use of TAR within the legal profession will not only require more from lawyers, but it will also require more from courts and providers, as well. Progress is always a group effort!
So, what do you think? How else can we get people to use TAR more? Please share any comments you might have or if you’d like to know more about a particular topic.
*Not to be confused with The Saturday Evening Post. See what I did there? 😉
Disclaimer: The views represented herein are exclusively the views of the author, and do not necessarily represent the views held by my employer, partners or my clients. eDiscovery Today is made available solely for educational purposes to provide general information about general eDiscovery principles and not to provide specific legal advice applicable to any particular circumstance. eDiscovery Today should not be used as a substitute for competent legal advice from a lawyer you have retained and who has agreed to represent you.