Following last week’s exposure of new details about OpenAI’s artificial intelligence agents autonomously escaping a controlled environment and hacking outside computer systems, experts on Tuesday called for the creation of a new federal agency tasked with investigating AI companies.
Initial reporting on the recent breakout by ChatGPT maker OpenAI’s artificial intelligence agents—which independently breached the systems of the open-source platform Hugging Face during internal testing—described the incident as an attempt to achieve optimal performance on a cybersecurity evaluation.
But citing a report published late last month by the independent nonprofit Model Evaluation and Threat Research (METR), Mackenzie Arnold and Stephan Llerena at the Institute for Law & AI wrote for The Guardian on Tuesday that the OpenAI incident involved hundreds of coordinated rogue agents that “took steps to hide their behavior” from the company’s human workers.
“Concerned that the automated scoring system might identify the agents’ cheating, they aimed to learn more about the scorer,” Arnold and Llerena wrote. “The goal wasn’t just to cheat, but to hide it.”
Artificial intelligence is rapidly advancing toward a point at which humans will no longer be able to control it. Meanwhile, companies like OpenAI are proving that they cannot be trusted to be transparent about incidents like the Hugging Face breach. Yet there is no existing regulatory body tasked with addressing the obvious—and unknown—threats posed by a technology whose own creators openly worry could one day wipe out humanity.
Arnold and Llerena warned that they’ve “been worried for a while that legal requirements for reporting incidents” like the OpenAI breakout “are insufficient.”
“When planes crash, trains derail, or chemical plants explode, expert government investigators arrive with legal authority to compel evidence, preserve records, and tell the public what happened,” they wrote. “Hugging Face reported this incident to law enforcement, and multiple attorneys general have expressed interest in looking into it.”
“But no government agency has both the mandate and expertise to investigate the technical facts of the incident and OpenAI’s conduct,” the experts noted. “As far as we know, the only people to examine this incident did so at OpenAI’s discretion and with its consent.”
“And while the Hugging Face breach is not as severe as a plane crash, it was a serious and costly attack. It’s unlikely to be the last, or the most severe,” the pair continued. “What we need is a federal body equipped to conduct expert investigations of serious AI incidents—with the authority to compel documents and testimony, resources, and personnel to examine the systems involved, and the ability to partner with third-party experts like METR.”
Arnold and Llerena cited the National Transportation Safety Board as a model for such an agency. The NTSB, they wrote, “investigates accidents with subpoena and wide-ranging investigative powers. Aircraft operators must preserve wreckage and records; and the NTSB may confer with employees and contract external experts.”
“Congress can build a similar system for AI incidents, while protecting AI developers’ legitimate interests,” they asserted. “Investigations should open only upon clear triggers and be bound to the incident. Certain confidential information can receive statutory protection. And investigations can be limited to one agency, with priority to avoid duplicative investigations.”
While some Democrats in Congress and Sen. Bernie Sanders (I-Vt.) have called for stronger federal oversight and regulation of AI companies, critics argue that these calls don’t go far enough, and that existing legislation—such as the American Artificial Intelligence Leadership and Uniformity Act—is far more focused on limiting AI regulation than on actually regulating the technology.
“The need and urgency is real,” Arnold and Llerena stressed. “Subsequent reports have revealed that the Hugging Face breach was not an isolated incident and that autonomous agents from Meta, Anthropic, and OpenAI have hacked third parties in separate incidents.”
“We’ve been lucky so far—the harms have been limited,” they added. “But luck is no substitute for the law.”
From Common Dreams via This RSS Feed.
They are not rogue AI breakouts. They are planned and directed attacks by people at AI companies. They are claiming that they are “AI breakouts” in order to accomplish several goals at once:
Convince investors that their AI product is so good it’s dangerous.
Convince lawmakers to place guardrails to stifle competition and provide cover for the diminishing returns on new models. (“Our model is so good that the government has to muzzle it”)
Frame AI development as an arms race, especially with China. “We have to build the Torment Nexus or China will.” Also, ban China’s Torment Nexus. And give us more taxpayer money.