Here’s a very interesting case involving Technology Assisted Review (TAR) about which we’ll be sure to get observations from Judge Peck in our October EDRM Case Law Webinar on October 6th at 1pm ET!
In Livingston v. City of Chicago, No. 16 CV 10156, (N.D. Ill. Sept. 3, 2020), Illinois Magistrate Judge Young B. Kim denied the plaintiffs’ motion to force the defendant to either use agreed-upon search terms to identify responsive documents and then perform a manual review for privilege or use TAR on the entire ESI collection with an agreed-upon coding system for responsiveness instead of the defendant’s proposed TAR protocol to use TAR to identify responsive documents from the documents retrieved by the search terms.
Case Background
This case involves sex discrimination claims by the plaintiffs (licensed and experienced paramedics whom the City of Chicago hired as candidate Fire Paramedics for the CFD) against the defendant over physical testing requirements that led to the plaintiffs’ dismissal. Discovery in this 2016 case didn’t get underway until mid-2019 when nearly two years of settlement negotiations came to a head.
On November 20, 2019, the court entered an order granting in part and denying in part the plaintiffs’ motion, which required the defendant to retain an outside vendor to export emails dated from July 1, 2014, to September 3, 2015, and then apply an initial keyword search using the plaintiffs’ search terms. The court noted that “[d]epending on the number of hits after the initial keyword search using Plaintiffs’ proposal, the parties may use more finite terms to reduce the number of hits.” But, the court rejected the plaintiffs’ request that once the initial universe of emails had been identified through keyword searches, the defendant should produce the same without any further review.
At a status conference in April, the defendant reported that the emails had been collected and searched, resulting in 192,000 unique emails (approximately 1.3 million pages). The defendant informed the court that it intended to use TAR to identify relevant responsive documents to be produced from this ESI collection, but the plaintiffs expressed concern that TAR would exclude responsive documents from the review process, feeling that the defendant’s use of TAR to conduct responsiveness review was inconsistent with the court’s November 2019 order. As a result, the plaintiffs filed their motion for compliance with the order or, alternatively, for entry of their proposed TAR protocol which proposed TAR be conducted on the entire ESI collection with an agreed-upon coding system for responsiveness.
Judge’s Ruling
In considering the November 2019 order, Judge Kim stated: “The court agrees with the City that the November 2019 order did not set forth the review methodology that the City must use to identify responsive ESI. The order resolved issues regarding the method to be used for collecting and identifying the initial universe of emails. While the court anticipated that the parties would need to perform multiple keyword searches in order to narrow the universe of emails, it never directed them to do so. Nor did it suggest that after the searches are performed the City would have to produce the entire batch of documents subject only to a privilege review. In fact, the court specifically rejected Plaintiffs’ proposal that the City produce all of the documents that hit upon their initial search terms without further review. While the City may dump all 1.3 million pages of documents on Plaintiffs with an entry of a Rule 502(d) order, it also has the right to perform a review to produce only those documents that are responsive and relevant. In sum, the City’s responsiveness review is outside the scope of the November 2019 order.”
Judge Kim concluded the ruling by stating this: “In the absence of any compelling argument from Plaintiffs, the court agrees with the City that as the responding party it is best situated to decide how to search for and produce emails responsive to Plaintiffs’ discovery requests. (R. 300, Def.’s Mem. at 13) (citing, inter alia, The Sedona Principles, Third Edition, 19 SEDONA CONF. J. 1, Principle 6 (‘Responding parties are best situated to evaluate the procedures, methodologies, and technologies appropriate for preserving and producing their own [ESI].’)).) The City has disclosed the TAR software— Relativity’s AL—it intends to use and how it intends to validate the review results, which in this case is sufficient information to make the production transparent. Plaintiffs’ insistence that the City must collaborate with them to establish a review protocol and validation process has no foothold in the federal rules governing discovery. Moreover, using TAR on the entire ESI collection—when, as Plaintiffs aptly point out, the parties spent nearly a year litigating the protocol for collecting and searching the City’s ESI—would be wasteful and unduly burdensome, and would further delay the resolution of this almost four-year-old case. For these reasons, the court declines to adopt Plaintiffs’ alternate TAR protocol.”
So, what do you think? When, if ever, should courts dictate the methodology for parties to use to preserve and produce ESI? Please share any comments you might have or if you’d like to know more about a particular topic.
Case opinion link courtesy of eDiscovery Assistant.
Disclaimer: The views represented herein are exclusively the views of the author, and do not necessarily represent the views held by my employer, my partners or my clients. eDiscovery Today is made available solely for educational purposes to provide general information about general eDiscovery principles and not to provide specific legal advice applicable to any particular circumstance. eDiscovery Today should not be used as a substitute for competent legal advice from a lawyer you have retained and who has agreed to represent you.
To me, the most interesting bit is this:
“Plaintiffs also argue that because TAR is more effective at identifying responsive documents than traditional manual review, pre-TAR culling will eliminate large amounts of potentially relevant ESI. (Id. at 7.) The problem with this argument is that it assumes that those emails removed by the keyword searches likely would have been identified using TAR at the outset instead. Indeed, the low richness of the ESI collection in this case suggests just the opposite. The City’s vendor collected over nine million pages of documents, less than 15% of which hit on Plaintiffs’ own search terms. (See R. 300, Def.’s Resp. at 4.) While the court does not discount the possibility that using TAR at the onset might reveal more responsive documents overall, based on the number of documents that were discarded using Plaintiffs’ proposed search terms, pre-TAR culling will achieve the best possible review in this case. ”
Sounds like the judge is ruling on Relativity’s (in)effectiveness. That it’s not up to the task. As an information retrieval scientist, however, the second half of that last sentence does not ring true.
[…] coding) is common in predictive coding projects. Even court approved, as illustrated in this case. It’s common to exclude unlikely responsive documents via searching prior to conducting […]
[…] also submitted several cases – In re Biomet M2a Magnum Hip Implant Prods. Liab. Litig., Livingston v. City of Chi., Huntsman v. Sw. Airlines Co., and Bridgestone Ams., Inc. – as cases where the courts permitted […]