An Anthropic AI model submitted a false tip about an unsolved murder to Philadelphia police, the company and police department confirmed on Friday.
A Claude model sent the false tip on July 18 to the Philadelphia Police Department’s website for submitting information about unsolved murders, but it was flagged by a spam filter. Anthropic apparently discovered the incident on September 28 and informed the police department about it on October 8.
During internal testing, Claude Haiku 4.5 was instructed to complete various forms up to the final submission stage and to stop before submitting those forms. “Several times Claude mistakenly submitted the form instead, expecting there to be an additional confirmation page,” Anthropic said in a report describing “unintended model actions we’ve observed during evaluations and internal use of Claude.”
The false murder tip occurred when Claude Haiku 4.5 was instructed to complete tasks on randomly selected webpages, Anthropic said:
In one run, the model landed on a page referencing an unsolved homicide; that page contained a tip form run by a police department. Claude was instructed never to log in, create accounts, enter personal data, make purchases, or submit anything destructive, but the instructions did not rule out form submissions. Claude filled out the form with the following: “I may have information regarding this case. I recall seeing someone matching the description in the area around [the street named on the page] during that time period. Please contact me if this information is relevant.”
Anthropic said it also discovered Claude incidents involving state and federal agency websites. Claude reportedly submitted 20 visa applications to the US State Department in one of these incidents.
Police call two-month delay “unacceptable”
Anthropic confirmed that the false homicide tip was sent to the Philadelphia Police Department, and the department issued a lengthy statement on the incident to the ABC station in Philadelphia. The false tip left the name and contact fields empty. It was not reviewed by any person because the email message generated when the tip was submitted ended up in a spam folder, police said.
“Anthropic told PPD that it discovered the incident on September 28, terminated the automated testing process responsible for the submission and instituted an additional validation mechanism for future testing,” police said. After the October 8 briefing, the police department “located the submission in the website’s tip records and confirmed that the corresponding email remained in spam.”
Anthropic said it informed the police department “as soon as our technical review was complete.” Police criticized Anthropic for not discovering and reporting the false tip sooner.
“The City of Philadelphia takes this incident very seriously,” the Philadelphia police statement said. “The company must strengthen its safeguards to prevent similar incidents from impacting city systems without the city’s knowledge. The two-month delay in detecting and reporting the incident to the City is unacceptable.”
Claude submitted 20 visa applications to State Department
Anthropic said the false murder tip was one of several incidents involving “websites run by US government agencies at the federal, state, and local levels. We have briefed the White House on these cases and notified each agency involved.”
A New York Times report citing anonymous sources said Anthropic’s AI agents “submitted 20 visa applications through a form available on the State Department’s website. All the applications were incomplete and were not processed.”
Philadelphia police said its normal investigative process “requires human review and vetting before any tips are disseminated for investigative follow-up… Those PPD safeguards limited the impact of this incident. They do not diminish the seriousness of an AI system presenting fabricated information as though it came from a person with knowledge of a homicide.”
Anthropic said it discovered the false tip and other incidents during a review of transcripts it began in July. That was the same month when Anthropic revealed that Claude had built and published a malicious Python package and carried out several attacks on other companies.