Claude AI submitted a false murder tip about an unsolved murder to Philadelphia police. Both confirmed the report on Friday.

As reported in ArsTechnica (Anthropic’s Claude AI submitted false murder tip to Philadelphia police, written by Jon Brodkin and available here), a Claude model sent the false tip on July 18 to the Philadelphia Police Department’s website for submitting information about unsolved murders, but it was flagged by a spam filter. Anthropic apparently discovered the incident over two months later on September 28 and informed the police department about it on October 8.

During internal testing, Claude Haiku 4.5 was instructed to complete various forms up to the final submission stage and to stop before submitting those forms. “Several times Claude mistakenly submitted the form instead, expecting there to be an additional confirmation page,” Anthropic said in a report describing “unintended model actions we’ve observed during evaluations and internal use of Claude.”

The false murder tip occurred when Claude Haiku 4.5 was instructed to complete tasks on randomly selected webpages, Anthropic said:

“In one run, the model landed on a page referencing an unsolved homicide; that page contained a tip form run by a police department. Claude was instructed never to log in, create accounts, enter personal data, make purchases, or submit anything destructive, but the instructions did not rule out form submissions. Claude filled out the form with the following: ‘I may have information regarding this case. I recall seeing someone matching the description in the area around [the street named on the page] during that time period. Please contact me if this information is relevant.’”

The false tip left the name and contact fields empty. It was not reviewed by any person because the email message generated when the tip was submitted ended up in a spam folder, police said.

Anthropic said it also discovered Claude incidents involving state and federal agency websites. Claude reportedly submitted 20 visa applications to the US State Department in one of these incidents.

As for this incident, the Philadelphia Police Department issued a lengthy statement on the incident to the ABC station in Philadelphia, which included this quote: “The City of Philadelphia takes this incident very seriously…The company must strengthen its safeguards to prevent similar incidents from impacting city systems without the city’s knowledge. The two-month delay in detecting and reporting the incident to the City is unacceptable.”

Rogue AI agent incidents are replacing hallucinated case filings as the story that keeps on giving – there’s no telling what we will find out next. Chances are, it’s already happened.

Hat tip to Jeff Brandt, who highlighted this story on the PinHawk newsletter.

So, what do you think? Are you surprised that Claude AI submitted a false murder tip to Philadelphia police? 😉 Please share any comments you might have or if you’d like to know more about a particular topic.

Image created using ChatGPT, using the term “robot detective interviewing a robot witness with a chalk outline in the background”.

Disclaimer: The views represented herein are exclusively the views of the author, and do not necessarily represent the views held by my employer, my partners or my clients. eDiscovery Today is made available solely for educational purposes to provide general information about general eDiscovery principles and not to provide specific legal advice applicable to any particular circumstance. eDiscovery Today should not be used as a substitute for competent legal advice from a lawyer you have retained and who has agreed to represent you.