Two healthcare companies have been sued over their AI model because it’s alleged to have a 90% error rate to deny care.

According to two articles a few weeks apart from Beth Mole at Ars Technica (here and here), UnitedHealth and Humana are the two healthcare companies have been sued – in Minnesota and Kentucky respectively. Both cases are class actions. Both healthcare companies are allegedly using an artificial intelligence model with a 90 percent error rate to deny health coverage to elderly patients, forcing them to “drain their life savings” or leaving them with “overwhelming medical debt”.

Both cases are aimed at the insurer’s use of the AI tool nH Predict, which was developed by NaviHealth to forecast how long patients will need care after a medical injury, illness, or event. In both cases, the plaintiffs claim that the insurers use the “flawed” AI model to pinpoint the exact date to blindly and illegally cut off payments for post-acute care that is covered under Medicare plans—such as stays in skilled nursing facilities and inpatient rehabilitation centers. The AI-powered model comes up with those dates by comparing a patient’s diagnosis, age, living situation, and physical function to similar patients in a database of 6 million patients. In turn, the model spits out a prediction for the patient’s medical needs, length of stay, and discharge date.

But the plaintiffs argue that the model fails to account for the entirety of each patient’s circumstances, their doctors’ recommendations, and the patient’s actual conditions. And they claim the predictions are draconian and inflexible. For example, under Medicare Advantage plans, patients who have a three-day hospital stay are typically entitled to up to 100 days of covered care in a nursing home. But with nH Predict in use, patients rarely stay in a nursing home for more than 14 days before claim denials begin.

Though few people appeal coverage denials generally, of those who have appealed the AI-based denials, over 90 percent have gotten the denial reversed, the lawsuits say. However, according to one of the articles, “the win is short-lived—UnitedHealth will send new denials soon after, sometimes within days.”

Still, the insurers continue to use the model and NaviHealth employees are instructed to hew closely to the AI-based predictions, keeping lengths of post-acute care to within 1 percent of the days estimated by nH Predict. NaviHealth employees who fail to do so face discipline and firing, the lawsuits claim. The sub-heading of the first linked article above related to UnitedHealth says: “AI’s error rate is like a feature, not a bug.”

Overall, the patients claim that Humana and UnitedHealth are aware that nH Predict is “highly inaccurate” but use it anyway to avoid paying for covered care and make more profit. They claim the denials are “systematic, illegal, malicious, and oppressive.”

While Humana uses NaviHealth’s nH Predict, UnitedHealth owns them. UnitedHealth Group’s Optum business acquired them in May of 2020. Apparently, in October, they decided to discontinue the NaviHealth name “after the company faced congressional criticism over the use of its algorithms to cut off payments for patients’ care”.

It’s important to remember at this point that these are all allegations against the two healthcare companies in two class action lawsuits. Nothing is proven at this point. Still, what’s being claimed (and reported in the two articles) is serious stuff. These are the types of stories that give people pause about the ubiquitous encroachment of AI algorithms in our lives.

Hat tip to Tom O’Connor for the heads up on the second article and the issue overall!

So, what do you think? Are you concerned about AI models making decisions about our lives? Please share any comments you might have or if you’d like to know more about a particular topic.

Image created using GPT-4’s Image Creator Powered by DALL-E, using the term “robot doctors turning their backs on patients”.

Disclaimer: The views represented herein are exclusively the views of the authors and speakers themselves, and do not necessarily represent the views held by my employer, my partners or my clients. eDiscovery Today is made available solely for educational purposes to provide general information about general eDiscovery principles and not to provide specific legal advice applicable to any particular circumstance. eDiscovery Today should not be used as a substitute for competent legal advice from a lawyer you have retained and who has agreed to represent you.