I love it when Tom O’Connor gets into one of our topics and provides his own analysis! So, when he told me he was going to write about the blog post that Jim Gill wrote on the NIST study and the trustworthiness of AI, I expected he would have a lot of interesting takes on it.
Tom’s post on his Techno Gumbo blog (Is the New NIST Standard for AI Looking at the Wrong End of the Horse?) discusses the trustworthiness of AI (after switching analogies from the one referenced in the title – and today’s graphic – to one where he notes “that this is a case where we DO want to see the sausage being made”*) that “NIST talks about trust as a key element in getting lawyer buy in on AI. True, but first it would be nice if vendors explained to us what they actually mean by AI.”
Stating that “explaining how ‘revolutionary’ or ‘groundbreaking’ your AI is only helps me if I know specifically how it works in my particular use case”, Tom provides “some examples of what vendors say about AI right off their web sites”, as follows:
AI is the next frontier.
AI is the future of eDiscovery
Our AI uses cutting edge artificial intelligence and machine learning.
Our AI gives extraordinary results.
AI … Believe the Hype
Embrace the groundbreaking magic of artificial intelligence with (name deleted)
Precise predictions in a fraction of the time required for traditional review
Infusing AI across the entire E-Discovery process
(name deleted) artificial intelligence capabilities are built on top of the latest innovations in Deep Learning, Natural Language Processing, compute intensive hardware processing and other related architecture approaches (editors note: “compute” is not a typo)
And his personal favorite
yeah … this is what the future feels like
When it comes to the trustworthiness of AI, Tom nails it when he says, “part of the problem is that people aren’t really sure how these programs work”, which illustrates what a difficult subject it is. He also provides a link to the slide deck published by Tess Blair of Morgan Lewis (who co-presented the Ethics of AI presentation I covered last week, and many of the slides in the linked deck were in that presentation), provides an informative discussion of the comparison of AI to online music search engines Pandora and Spotify (a comparison which he “hates”), links to an article by Bob Ambrogi about a study by law librarians showing that different legal research platforms deliver surprisingly different results, goes into depth into The Art of “Thinking Like a Data Scientist” (which, it turns out, is really hard) and more.
Tom’s penultimate point is that “We have an ethical duty to truly understand this technology in order to be able to explain it to our clients and the Court, when required.” The American Bar Association actually “urges courts and lawyers to address the emerging ethical and legal issues related to the usage of” AI and has for nearly two years with Resolution 112, which was adopted in August 2019 (and covered by me here).
When it comes to the trustworthiness of AI, how do we get lawyers to trust it? I think (as Tom discusses) the responsibility lies both with the providers of the AI technology to make it more transparent and understandable, and the legal profession to try to understand it better. As Canned Heat sang in their 1970’s hit (which I suspect is in Tom’s playlists somewhere), Let’s Work Together!
So, what do you think? Can we improve the trustworthiness of AI (or at least get lawyers to trust it more)? Please share any comments you might have or if you’d like to know more about a particular topic.
* I could have used that analogy as today’s graphic. You’re welcome. 😉
Disclaimer: The views represented herein are exclusively the views of the author, and do not necessarily represent the views held by my employer, my partners or my clients. eDiscovery Today is made available solely for educational purposes to provide general information about general eDiscovery principles and not to provide specific legal advice applicable to any particular circumstance. eDiscovery Today should not be used as a substitute for competent legal advice from a lawyer you have retained and who has agreed to represent you.