AI cautionary tales are plentiful these days! This AI plush toy exposed thousands of private chats with children – for anyone with a Gmail account to read!

As discussed in Malwarebytes (An AI plush toy exposed thousands of private chats with children, written by Pieter Arntz and available here), Bondu’s AI plush toy exposed a web console that let anyone with a Gmail account read about 50,000 private chats between children and their cuddly toys.

Bondu’s toy is marketed as:

“A soft, cuddly toy powered by AI that can chat, teach, and play with your child.”

What it didn’t say is that anyone with a Gmail account could read the transcripts from virtually every child who used a Bondu toy. Without any actual hacking, simply by logging in with an arbitrary Google account, two researchers found themselves looking at children’s private conversations.

While Bondu put “18 months of beta testing with thousands of families” into its product to ensure its toy didn’t veer off into “unsafe or inappropriate behavior” (like this one did when it quickly veered from friendly chat into sexual topics and unsafe household advice), they apparently missed the fact that the company’s public-facing web console allowed anyone to log in with their Google account. The chat logs between children and their plushies revealed names, birth dates, family details, and intimate conversations. The only conversations not available were those manually deleted by parents or company staff.

As the article notes, “these chat logs could (have) been a burglar’s or kidnapper’s dream, offering insight into household routines and upcoming events.”

No kidding.

According to the article, Bondu took the console offline within minutes of disclosure, then relaunched it with authentication. The CEO said fixes were completed within hours, they saw “no evidence” of other access, and they brought in a security firm and added monitoring.

Kudos to them for responding quickly. Still, this latest cautionary tale shows that You can really put a lot of effort into creating a secure and safe product but miss one little thing that can undermine it all. Be careful out there!

So, what do you think? Do you have any other AI cautionary tale stories to add? Please share any comments you might have or if you’d like to know more about a particular topic.

Image created using Microsoft Designer, using the term “robot plush toy speaking to a human kid”.

Disclaimer: The views represented herein are exclusively the views of the authors and speakers themselves, and do not necessarily represent the views held by my employer, my partners or my clients. eDiscovery Today is made available solely for educational purposes to provide general information about general eDiscovery principles and not to provide specific legal advice applicable to any particular circumstance. eDiscovery Today should not be used as a substitute for competent legal advice from a lawyer you have retained and who has agreed to represent you.