It may be the most comforting phrase in AI. But the idea of human in the loop is not enough to ensure success, given human challenges.

According to Jerry Lawson in LLRX (available here), “Human in the loop” may be the most comforting phrase in the AI business; unfortunately, the human is more likely to be answering email, running late for lunch, and reviewing output number 37 of 40.

What could go wrong? As Lawson notes, a lot.

• Automation Bias. This is what I’ve been saying. AI has uncovered one of the oldest tricks in professional life: people are more likely to believe something when it is neatly formatted. We all tend to trust outputs that look polished, confident, and complete. There are 1,812 examples of that and counting as of this morning.
• Cognitive Overload. Billing in six-minute increments makes it difficult, if not impossible, to verify every AI work product.
• The Dashboard Illusion. The reviewer may see a tidy paragraph and an “Approve” button. What the reviewer may not see is which sources the system ignored, what assumptions it made, or what it did five steps earlier. The human is “in the loop,” but lacks the information needed for effective review.
• Speed Asymmetry. AI systems generate outputs and take actions faster than humans can meaningfully evaluate them.

If one or more of these issues get in the way, human in the loop could become “scapegoat in the loop”, as it has happened in the 1,812 examples referenced above.

Lawson provides a terrific discussion of what’s required for effective human review as follows:

• Time: Reviewers can’t be expected to rush through forty decisions before lunch.
• Visibility: The system must expose sources, assumptions, significant actions, and uncertainties.
• Authority: The reviewer must be able to stop or reverse the process without being punished for slowing things down
• Manageable volume: No review design can survive output volumes beyond realistic human attention.

Finally, independence and incentives matter. When we grade a lawyer’s performance by how quickly they process approvals, they may technically have the authority to reject an output but have little practical incentive to exercise it.

As Lawson notes in concluding his excellent article: “Sometimes ‘human in the loop’ doesn’t function as a safety mechanism. It’s more like a liability arrangement. The machine does its work, and the human clicks ‘approve.’ When that happens, we all know whose name will appear in the disciplinary opinion.”

People are very quick to put the brakes on agentic AI, shouting from the rooftops that “we must have a human in the loop!” But the mere idea of human in the loop is not enough to ensure success. The humans in the loop need to be set up for success in terms of training, manageable workloads and reasonable expectations. We have 1,812 examples to learn from – and counting.

So, what do you think? How is your organization managing its human in the loop processes? Please share any comments you might have or if you’d like to know more about a particular topic.

Image created using DALL-E 3, using the term “robot goat wearing a business suit in a courtroom”.

Disclaimer: The views represented herein are exclusively the views of the author, and do not necessarily represent the views held by my employer, my partners or my clients. eDiscovery Today is made available solely for educational purposes to provide general information about general eDiscovery principles and not to provide specific legal advice applicable to any particular circumstance. eDiscovery Today should not be used as a substitute for competent legal advice from a lawyer you have retained and who has agreed to represent you.