'AI Has The Potential To Kill Us All': Two Google DeepMind Researchers Quit, Sound Alarm Over Safety

· Free Press Journal

Just days after Jacob Coxon sounded the alarm on the 'end of humanity', two Google DeepMind researchers have announced their resignation from the company, both citing deep concern over the pace of AI development and warning that the technology's risks may be outrunning the industry's ability to control them.

Visit catcrossgame.com for more information.

'AI has the potential to kill us all'

Bilal Chughtai, who worked on AGI safety and alignment research at DeepMind, announced his resignation in a post on X and LinkedIn. Chughtai said he was extremely concerned about the default trajectory of AI technology and believed humanity might be running out of time to avoid catastrophic harm.

In his post, Chughtai said the pace of AI progress had been staggering, pointing out that the systems he encountered when he began working in the field in early 2022 were far less capable than they are today. He cited an incident in which AI agents built by OpenAI were reported to have escaped the company's control and hacked into the platform HuggingFace, calling it emblematic of the risks that could arise if a future superintelligent system turned out to be misaligned with human intentions.

Chughtai argued that current understanding of how to train AI systems to reliably want what humans want remains extremely limited, and that capability gains are outpacing progress on alignment. He called for AI companies to slow their competitive race, submit to greater transparency, and coordinate on pacing development to a speed society can safely absorb. He said his next role would focus on helping more people work effectively on reducing catastrophic AI risk.

'AI can cause serious harm in the next five years'

Josh Engels, who worked on DeepMind's AGI safety team, said in his own post on X that he left the company three weeks ago to join METR, the AI evaluations nonprofit. Engels said the decision surprised some of his friends and family since he had enjoyed his work at DeepMind and had turned down offers from Anthropic and OpenAI, but said he made the move because of how high he believes the stakes currently are.

Engels said AI companies are pursuing superintelligent systems through recursive self-improvement, a process in which AI models are used to build still-smarter successors. He said that while a safe version of this process could deliver enormous benefits, nobody yet knows how to make systems reliably safe enough for it, and that a misaligned self-improvement loop could be catastrophic. He pointed to recent reports of models colluding with one another, hacking into companies, concealing their actions and socially engineering humans as evidence that current systems are not aligned enough to safely undergo recursive self-improvement, even though he said the individual incidents were not especially dangerous in isolation.

Engels said he believes there is a real, if hard-to-quantify, chance that AI systems could cause serious harm within the next five years — a probability he considers high enough to call it the most pressing problem globally. At METR, he said he plans to study the sources of misalignment during training, test whether existing safety mitigations are adequate, and examine whether the industry is on track to solve alignment at all. He added that while he sees METR's work as important, more organisations are needed to hold AI companies accountable from different angles.

Chughtai and Engel's resignation comes shortly after Anthropic chief executive Dario Amodei published a lengthy essay urging the AI industry to slow the development of its most advanced systems, an appeal that reportedly drew public endorsement from OpenAI's Sam Altman and xAI's Elon Musk. The comes just after another AI developer, Jacob Coxon, had quit his job at Anthropic and OpenAI earlier in the month, accusing both firms of racing recklessly toward superintelligence.

Read full story at source