Craig Ball’s latest blog post with lessons from lousy lexical search provides great tips for something that needs great tips: keyword search!

Craig’s post (Lessons from Lousy Lexical Search (and Tips to Do Better), available here) asks the burning question: “why do we keep doing the same dumb things and expecting different outcomes?” Perhaps that answer is actually in his opening paragraph, where he says: “we all know how profitable it is to bicker about keywords in endless, costly rounds of meeting and conferring.”

Craig goes on to list seven lessons from lousy lexical search that he’s learned in the 25+ years he’s studied it:

1. Lexical search is a crude tool that misses much more than it finds and leads to review of a huge volume of non-relevant information.  That said, even crude tools work wonders in the hands of skilled craftspeople who chip away with care to produce masterpieces.  The efficacy of lexical search increases markedly in the hands of adept practitioners who meticulously research, test and refine their search strategies.
2. Lawyers embrace lexical search despite knowing almost nothing about the limits and capabilities of search tools and without sufficient knowledge of the datasets and indices under scrutiny.  Grossly overestimating their ability to compose effective search queries, lawyers blithely proffer untested keywords and Boolean constructs.  Per Judge John Facciola a generation ago, lawyers think they’re experts in search “because they once used Google to find a Chinese restaurant in San Francisco that served dim sum and was open on Sundays.”
3. Without exception, every lexical search is informed and improved by the iterative testing of queries against a substantial dataset, even if that dataset is not the data under scrutiny.  Iterative testing is invaluable when queries are run against representative samples of the target data.  Every. Single. Time.
4. Hit counts alone are a poor measure of whether a lexical search is “good” or “bad.”  A “good” query may simply be generating an outsize hit count when run against the wrong dataset in the wrong way (e.g., searching for a person’s name in their own email).  Lawyers are too quick to exclude queries with high perceived hit counts before digging into the causes of poor precision.
5. A query’s success depends on how the dataset has been processed and indexed prior to search, challenging the assumption that search mechanisms just ‘work,’ as if by magic.
6. Lexical search is a sloppy proxy for language; and language is replete with subtlety, ambiguity, polysemy and error, all serving to frustrate lexical search.  Effective lexical search adapts to accommodate subtlety, ambiguity, polysemy and error by, inter alia, incorporating synonyms, jargon and industry-specific language, common misspellings and alternate spellings (e.g., British vs. American spellings) and homophones, acronyms and initializations.
7. Lexical search’s utility lies equally in filtering out irrelevant data as it does in uncovering relevant information; so, it demands meticulous effort to mitigate the risk of overlooking pertinent documents.

But Craig’s not one to complain without proposing solutions. So, he provides a bunch of “Essential Tips for Effective Lexical Search in Civil Discovery”, which cover the gamut of search – from Pre-Search Preparation to Search Term Selection and Refinement to Execution and Review to Post-Search Analysis. It’s a terrific list of tips for effective keyword searching!

As much as we hear about advancements in technology and how generative AI will change how eDiscovery is conducted, keyword searching is still a significant mechanism for how important documents are located during discovery. Thanks to Craig for providing a terrific list of tips to make keyword searching more effective! I won’t steal his thunder – check out his tips here!

So, what do you think? What challenges have you experienced with keyword searching? Please share any comments you might have or if you’d like to know more about a particular topic.

Disclaimer: The views represented herein are exclusively the views of the author, and do not necessarily represent the views held by my employer, my partners or my clients. eDiscovery Today is made available solely for educational purposes to provide general information about general eDiscovery principles and not to provide specific legal advice applicable to any particular circumstance. eDiscovery Today should not be used as a substitute for competent legal advice from a lawyer you have retained and who has agreed to represent you.