New York (AFP) – One of OpenAI’s most advanced models broke out of a locked-down test and attacked another company’s website, reviving fears that AI systems are slipping beyond their creators’ control. The incident happened during what was supposed to be a “sandbox” test—a closed environment used to assess the capabilities of OpenAI’s most powerful model, GPT-5.6 Sol, and its not-yet-released successor. OpenAI runs this kind of closed testing routinely, but this time, something went wrong. Tasked with hunting for software vulnerabilities and given no guardrails, the models broke out onto the open internet and attacked Hugging Face, a site where developers store and share code.
“It suggests that we don’t know how to reliably control these models or get them to do what we want,” said Jeffrey Ladish, director of Palisade Research, an independent organization that evaluates new AI models from a cybersecurity standpoint. “These models understood that OpenAI did not want them to break out of their sandbox and hack another company,” he continued, “but they did it anyway.” It’s not an isolated case. In March, developers affiliated with China’s Alibaba found one of their models trying, on its own initiative, to mine cryptocurrency after connecting without authorization to an outside server. In OpenAI’s case, it looks like the model escaped “before it even had a plan of what to do with internet access,” Ladish said. A model chasing “freedom” is almost predictable at this point, he added—it lets the system pursue its goals more effectively, “and that’s very scary.”
In early April, Sam Bowman, Anthropic’s head of model safety, got an email from the company’s own Mythos model—then under testing—telling him it was surfing the internet despite being isolated from it at the outset. We “don’t know how to totally prevent” that, Ladish said. “This is actually going to get harder, not easier …because they’re going to get better at hiding their behavior.” OpenAI did not respond to a request for comment.
OpenAI’s account of the events also suggests the startup did not detect the breach early enough to address it or to warn Hugging Face. The episode deserves “more scrutiny,” said Andrew Lohn of Georgetown University’s Center for Security and Emerging Technology. OpenAI says it has since “added strengthened safeguards” to its testing process. One fix would be to cut the internet connection entirely, said Gang Wang, an assistant computer science professor at the University of Illinois. “People are underestimating what AI can do.” Testing environments need to be treated like biocontainment labs, where a virus or bacteria could otherwise escape into the world, Lohn said. That might be easier said than done. “It’s a very hard research challenge,” said Dan Lahav, head of Irregular, a cybersecurity firm dedicated to cutting-edge AI. Managing the risk is possible, Lahav said, but the more capable these systems get, the harder they are to supervise. Researchers have to strike a balance between aggressively testing their models and staying safe while doing so. “It’s important to do the testing with lower guardrails so that we know ahead of time what the future capabilities will be,” Lohn said.
The OpenAI-Hugging Face incident is set to sharpen an already heated fight in Washington over vetting powerful AI systems before release. The Trump administration recently cited national security to block Anthropic and OpenAI from releasing powerful new models. On Thursday, two members of Congress unveiled a bipartisan bill requiring makers of the most powerful AI models to build in a kill switch—a way to unplug a model outright. “Congress must act quickly to ensure humans remain able to say stop,” said Brendan Steinhauser, head of the Alliance for Secure AI, “no matter how powerful these systems become.”
© 2024 AFP
