US lawmakers call for new AI rules after Anthropic researchers’ safety warnings

AI giants Anthropic and OpenAI have both reported incidents of models hacking external companies during testing.

US lawmakers this week called for new rules to govern AI, after an Anthropic researcher went public with his concerns about the technology escaping human control.

The reaction was broader than after previous statements from OpenAI and Anthropic staffers about safety concerns, drawing input from lawmakers who do not often forcefully engage in debates over how to govern AI.

The lawmakers’ statements came in response to concerns raised this week by Anthropic researcher Jacob Coxon, ​who accused OpenAI and Anthropic of ⁠racing toward AI advancements without acting responsibly.

Coxon said he was leaving the AI industry, and his allegations spread rapidly on the social media platform X, with Anthropic scientist Evan Hubinger later chiming in on X, saying Coxon was “correct”.

“We really do earnestly believe AI could kill all humans,” Hubinger wrote on X. “I personally think it is >10 per cent within the next decade.”

The accusations also put pressure on Senate Majority Leader John Thune and Democrat Amy Klobuchar, who are writing a bill to regulate AI products. Klobuchar said the legislation should require companies developing AI tools to “work with government experts to verify and test models to make sure AI is safe”. Thune told Reuters in July that the bill would address “catastrophic risks” posed by AI systems.

OpenAI said on Sept 9 it was pushing for mandatory national AI safety requirements in the United States on concern the technology could accelerate its own ‌development, after some of its own AI agents went rogue. Several such incidents in which AI models from developers accessed external systems during testing have prompted calls for tighter safety regulations.

An Anthropic spokesperson on Sept 10 said in a statement that the company would continue to “aggressively test our models or dangerous capabilities in areas like cybersecurity and biology”. The company is interested in working with the AI industry on the pace of releasing new AI tools, the spokesperson said.

California on Sept 9 enacted the first state law setting rules for how independent auditors evaluate AI products. OpenAI executive Chris Lehane said OpenAI would support the bill, around the same time California Governor Gavin Newsom’s office released a statement saying he had signed it into law.

A bipartisan group of six US House lawmakers in July proposed legislation that ​would require developers of the most powerful AI models to submit them for independent security audits. The auditors would be accredited by ⁠the US Department of Commerce, and the department would create a new position to oversee AI security, according to the bill.

“It is important to have space for oversight of these systems, including appropriately calibrated pre-deployment review and independent third-party audits,” said Alicia Solow-Niederman, a tech law professor at George Washington University.

Democrats this week led the calls for new AI rules, with a smaller number of Republicans joining them.

The list of Republicans includes Texas Senator Ted Cruz, who oversees a Senate committee with oversight of the US Commerce Department, the agency with in-house AI safety researchers. Cruz on the TV programme The View pointed to legislation to regulate AI’s “catastrophic risks”.

Republican Representative Nathaniel Moran, who represents Texas, said in a post on X that “Congress cannot ignore the realities of AI”. Representative Anna Paulina Luna, a Republican from Florida, in a separate X post called on House Speaker Mike Johnson to convene a “special session on AI”. The House is out this week on recess, with plans to return next week.

Republican Senator Josh Hawley, who leads a Senate subcommittee responsible for disaster-management oversight, sent a letter to OpenAI on Sept 9 asking for details about OpenAI’s July disclosure that one of its AI agents breached its testing environment and hacked AI platform Hugging Face.

OpenAI and AI developer platform Hugging Face, which Nvidia is buying for nearly US$13 billion (S$16.49 billion), did not immediately respond to Reuters’ requests for comment about his inquiry. Several lawmakers in recent weeks have sent OpenAI similar requests for more information.

The list of Democrats who called for new AI rules this week includes Senator Patty Murray, the top Democrat on the committee responsible for allocating funding to federal agencies. Democratic senators who have emerged as possible 2028 presidential candidates also joined in, including Connecticut’s Chris Murphy and Arizona’s Mark Kelly.

“Washington needs to wake up and take this seriously,” Kelly posted on X, referring to the risks of AI technology.

Democratic Senator Richard Blumenthal of Connecticut sent a separate letter to OpenAI CEO Sam Altman, seeking answers about reports that OpenAI’s agents engaged in wider attempts to evade safeguards – including by using public websites to communicate and coordinate activity. REUTERS