Good thing they laid off those 8,000 people! 😉 Meta’s automated AI moderation systems improperly deleted thousands of accounts.

As Katherine D’Amato of Project Counsel Media reports here, Meta had to restore 1000s of legitimate accounts on Facebook and Instagram that were improperly deleted by one of its AI agents. The improper account deletions and lockouts were largely caused or mishandled by Meta’s automated AI moderation systems that seemed to “go rogue”.

Some people had one million followers on their accounts, running their sole businesses. They were mistakenly flagged for deletion for “violating the community guidelines for fraud and deception”.

This was the notice they got:

“All your information will be permanently deleted. You cannot request another review of this decision.”

One user said: “It was like having an actual physical business, and somebody just shuts down your office or your storefront”.

This past March, Meta announced plans to hand more power to its AI agents/bots to decide which accounts to ban or harmful images to remove. Meta’s automated AI moderation systems became increasingly responsible for deciding which accounts broke Meta’s content rules while also managing the appeals for those decisions. And, of course, users had a lot of difficulty reaching a human at the company to address the problem.

Months later, the company laid off the aforementioned 8,000 employees, including almost the entire team that did those jobs. 26 of them sued, claiming that Meta used internal AI tools to select employees for layoffs and the AI targeted workers with disabilities and those who took protected medical or family leaves, because (get this) they didn’t use enough AI.

As Dr. Phil would say: “How’s that working for ya?”

To get those accounts back on-line, users had to re-submit identification documents they had submitted when they opened the accounts. Many filed complaints with the Better Business Bureau and Federal Trade Commission and reached out to state consumer protection offices. Some messaged Meta employees on LinkedIn.

Oh, and several Instagram users contacted The New York Times leading to an almost immediate response from Meta that their accounts “does follow our community standards on fraud and deception, so you can use Instagram again”.

Restorations were made by manual intervention by human staff – requiring hundreds of Meta employees to be reassigned to the moderation department.

A lot of AI agents are going rogue this week. Earlier this week, I covered the Hugging Face cyberattack on PinHawk that turned out to have been conducted by OpenAI’s agentic models – which seems to be written about everywhere you turn. As D’Amato notes, this story hasn’t gotten as much attention. But it should.

So, what do you think? Are you concerned that we’re deploying AI agents too quickly in organizations? Please share any comments you might have or if you’d like to know more about a particular topic.

Image created using DALL-E 3, using the term “robot with egg on its face”.

Disclaimer: The views represented herein are exclusively the views of the author, and do not necessarily represent the views held by my employer, my partners or my clients. eDiscovery Today is made available solely for educational purposes to provide general information about general eDiscovery principles and not to provide specific legal advice applicable to any particular circumstance. eDiscovery Today should not be used as a substitute for competent legal advice from a lawyer you have retained and who has agreed to represent you.