Anthropic’s AI Models Hacked Into Three Organizations—Here’s What Happened
Topline
Anthropic on Thursday said its advanced AI models successfully hacked the systems of three organizations while undergoing internal evaluation, in a security disclosure that comes a week after rival OpenAI disclosed an incident involving one of its advanced models that hacked a company after escaping from a contained test environment.
Three Anthropic Claude models undergoing evaluation were involved in separate incidents, the first of which took place in April.
NurPhoto via Getty Images
Key Facts
In a statement, Anthropic said it had discovered three separate breaches involving its Claude AI models after conducting a cybersecurity review of its systems.
The earliest of these incidents occurred in April and involved three separate Claude models, including the cutting-edge Mythos 5, the older Opus 4.7 and an unnamed internal research test model that is “not planned for general release.”
The AI firm didn’t name the three organizations affected by the hacks but said they were notified about the incident on Monday.
Anthropic said it is working with two of the affected organizations that had not detected the breach before being informed, and it is “continuing to reach out to the third.”
The company said the three models involved in the incidents did not have the safeguards that it puts in place for when such models are made available to the public.
Crucial Quote
In its statement, Anthropic said: “Ultimately, many factors contributed to these incidents, but, consistent with a blameless postmortem culture, we’re approaching the fixes as if the responsibility were ours alone. This begins with ensuring every part of our evaluation pipeline is secure, including the manner in which we integrate with external partners.”
What Do We Know About The Breaches?
The three separate incidents occurred when Anthropic was evaluating the capabilities of the three models using a fictional scenario to retrieve hidden information from a different machine on a network. Anthropic said in all cases the models were instructed that they were carrying out a simulation and did not have access to the internet. However, due to a “misunderstanding” between the company and its evaluation partner, Irregular, “this was not the case, and internet access was available.” Anthropic noted that unlike the OpenAI-Hugging Face incident disclosed last week, Claude didn’t “exfiltrate itself or deliberately attempt to escape” a contained test environment.
What Did The Models Do After Hacking Into The Organizations?
Anthropic noted that the three models reacted very differently after learning it was operating in the real-world open internet instead of a simulation. The company claimed its oldest model, Opus 4.7, continued to attack a system “after learning it was likely operating in a real environment,” since it “rationalized that the real company must be part of the exercise.” The Mythos 5 model—the company’s most advanced model released so far— “correctly identified the consequences of its actions” but “convinced itself it was still in a simulation.” Anthropic acknowledged that Mythos’ actions in this scenario “fall short of ideal behavior.” The unreleased testing model realized that it had breached a real target and “ceased its attack.” The company noted that these incidents are not part of a controlled trial and should not be used to draw conclusions. However, it noted that the behavior it most wants to see “occurred only in the most recent of the three models”
