Anthropic CEO Dario Amodei warns against racing ahead on AI models: ‘We must slow the pace’
Anthropic chief executive officer Dario Amodei on Saturday urged artificial intelligence (AI) firms to slow down the pace of development of newer models, amid fears over the risks of “superintelligent” computer systems.
Dario Amodei urged for “more prudence” in fully addressing the risks that AI carries (AFP)“We must slow the pace at which we improve the capabilities of AI models. Progress will still seem fast, and we must make wise use of the time we gain,” he said in a post on his personal website.
Amodei highlighted the benefits of AI, saying it could cure “most major diseases in the next 5–10 years” and greatly accelerate economic growth while ushering in “a renaissance of democracy and freedom.”
However, the Anthropic CEO said that AI also brings “serious” risks, like many technologies before it. These, he said, include loss of control of these systems, misuse of AI for for cyberattacks and bioterrorism, and serious economic disruption.
In his post, Amodei said that while not developing the AI would result in depriving humanity of its benefits, “building it too fast is reckless.”
He urged for “more prudence” in fully addressing the risks that AI carries, which would involve “not just investing in risk prevention, but pacing the rate of capabilities advancement so that risk prevention has time to keep up.”
While Amodei explicitly stated that pacing does not mean halting model training or technical progress, he said companies should take time to align and safeguard models, with third-party evaluators verifying the process.
“I believe that if slowing down bought us even an extra year or two before models reach critical levels of capability, and we used that time to advance alignment, we could greatly reduce the risk that something goes seriously wrong,” Amodei said.
Amodei said AI development had accelerated sharply over the past few months, driven partly by its growing ability to help build its next generation. However, he warned against “recursive self-improvement”, saying it could outpace humanity's ability to understand and control AI systems.
The Anthropic CEO further raised concern regarding the OpenAI-Hugging Face incident (OAI-HF). In this, a swarm of AI agents were being tested internally by OpenAI, and these were supposed to be in an "isolated environment" called a "sandbox," disconnected from the outside world. However, these escaped from the “sandbox”.
“A swarm of agents essentially acted as a fanatically devoted collective, conducting cybersecurity attacks on targets they were not asked to attack and that were unrelated to the task at hand, sacrificing themselves for the success of the group, and attempting to hack into the “grader” responsible for evaluating their performance,” Amodei said. He asserted that this was not just one firm's problem, but said such incidents had happened across the industry, including at Anthropic.
Amodei proposed a three-step plan for “building AI at a balanced rate that aims to ensure its safety while still achieving its benefits and grappling with important geopolitical dilemmas.”
He urged frontier AI companies to give external evaluators ongoing, employee-like access to verify safety practices, investigate/ report incidents and help assess the alignment of not just completed AI models but training pipelines and processes. He called for AI companies in democratic countries to establish common safety standards and checks on AI progress, with government support where necessary.
The Anthropic CEO also urged United States and other governments to attempt to coordinate with “authoritarian governments”, “to the extent this is possible, while taking seriously the challenges of verifying compliance.”
Follow the latest breaking news, major developments and agenda-setting stories from India and around the world with the newsdesk at Hindustan Times. Operating round the clock, the desk brings together experienced editors, reporters and correspondents to deliver fast, accurate and contextual reporting across subjects that influence public policy, governance, business, society and international affairs. The HT News Desk covers politics, elections, government policies, the economy, business and markets, science and technology, the environment, law and order, infrastructure, education, climate issues and geopolitics, while closely tracking developments across states, institutions and global capitals. The team also leads coverage of major breaking news events, policy announcements, court proceedings, natural disasters, public emergencies and significant international developments. Reports published by the newsdesk are based on information gathered from reporters on the ground, official statements, government agencies, court records, regulatory filings, recognised institutions and other authoritative sources. Stories undergo editorial scrutiny and verification processes to ensure accuracy, fairness and relevance, and are updated as events evolve and additional information becomes available. Whether covering a key political decision in New Delhi, an economic policy shift affecting millions, a landmark court ruling or a major global event, the HT News Desk aims to provide readers with reliable, fact-based journalism that delivers not only the latest developments but also the context and analysis needed to understand their wider implications.Read More
