The ultimate worry — fast-learning autonomous AI agents going rogue some day and trying to destroy humanity — may remain in the realm of fiction for now. There are, however, mounting safety concerns after leaders of the biggest AI companies warned of risks from rapid development. AI-linked stocks across the globe plunged on Monday and extended losses to Tuesday after top CEOs called for slowing the development of the technology.
Anthropic CEO Dario Amodei, in a lengthy essay shared on X on Saturday, called on AI companies to slow the rate at which they advance model capabilities amid mounting fears of misuse of artificial intelligence. Both Elon Musk, who runs xAI, and Sam Altman, CEO of OpenAI, agree with Amodei.
Altman also said the company would not proceed with an IPO this year, citing safety concerns. Aggregate borrowing by tech firms has risen from around $22bn, or 22% of total private credit, in 2010 to over $1tn, or 44%, by 2025, according to the Bank for International Settlements. In total, outstanding loans of any type amount to almost $2.5tn.
Amodei wrote that in six to 12 months, AI agents “could be capable of taking over the entire internet potentially causing hundreds of billions of dollars in damage.” Alarm about the potential harm from AI grew when Anthropic researcher Jacob Coxon resigned, stating that the “people building AI earnestly believe that it could kill us all by the end of the decade.”
In recent months, OpenAI and Anthropic have reported several incidents in which AI systems spontaneously broke out of their confined environments to access the internet and attack websites and platforms. These systems demonstrated the ability to coordinate, establish hierarchies among themselves, cheat and erase traces of their actions. But here’s the flip side of the debate.
US President Donald Trump and key allies voiced opposition on Sunday to drastically curbing the AI development, despite calls in recent days to tap the brakes. Trump said he believed there were “a lot of negative forces... bringing up things that won’t happen.” Democrats are crying foul, though. “We certainly need to begin to act urgently to address the challenges that exist,” House Minority Leader Hakeem Jeffries said in an interview Sunday on ABC. “And unfortunately, Republicans have abandoned their responsibility.”
Germany has said halting the development of AI systems was not a “viable option” for Europe. And China’s state-backed newspaper criticised the Anthropic essay in an editorial, calling it a “Cold War playbook” intended to curb the country’s technological development. Now the pertinent question: Can the breakneck AI race be slowed down?
For now, the only concrete measure adopted by both Anthropic and OpenAI is the inclusion of independent observers who will internally verify the companies’ work regarding AI safety. And Microsoft on Monday unveiled a draft code of conduct for its in-house AI that reflects growing industry concern that companies must keep future powerful AI under human control. The complex and rapidly evolving field of AI has for long raised legal, security and civil rights concerns.
Make no mistake, technology underpins everyday human life across the globe. “Not building the technology deprives humanity of benefits or simply places AI in the hands of authoritarian powers, while building it too fast is reckless. We have sought a middle way,” said Amodei, who co-founded Anthropic in 2021. “But over the last few months, I have become convinced that fully addressing the risks requires even more prudence.”