AI Engineers Are Having Their Oppenheimer “Destroyer of Worlds” Moment

“Now I am become Death, the destroyer of worlds.” These lines from the Bhagavad Gita echoed through the mind of Robert Oppenheimer at 5:29 a.m. on July 16, 1945 when he watched the first detonation of a nuclear bomb in the New Mexico desert—or so he later claimed in the 1965 NBC News documentary The Decision to Drop the Bomb. “We knew the world would not be the same. A few people laughed; a few people cried. Most people were silent,” Oppenheimer said to the camera. “I remembered the line from the Hindu scripture, the Bhagavad Gita; Vishnu is trying to persuade the prince that he should do his duty, and to impress him, takes on his multiarmed form and says, ‘Now I am become Death, the destroyer of worlds.’ I suppose we all thought that, one way or another.” Read more: “The Day Oppenheimer Feared He Might Blow Up the World” Oppenheimer was later deeply tormented by the human consequences of his work with The Manhattan Project. In a speech at Los Alamos in October 1945, just a few months after Hiroshima and Nagasaki were leveled by nuclear strikes, killing tens of thousands of people instantly, he called the bomb “an evil thing.” Several years later, in 1949, as head of the General Advisory Committee for the Atomic Energy Commission, Oppenheimer fiercely opposed the development of the hydrogen bomb, arguing that it was a weapon of genocide that could unleash total global destruction. Today, scientists who work on AI seem to be having their own Oppenheimer moment. This week, Jacob Coxon, an AI researcher who left Anthropic earlier this week so that he could express his concerns, wrote on X/Twitter that “the people building AI earnestly believe it could kill us all by the end of the decade. This is not a marketing stunt.” In a longer thread, he said companies are still building AI despite this existential risk because they haven’t truly internalized the stakes and they’re in a race to get there first. A chorus of others, many of them people working at Anthropic, voiced similar sentiments. “Jacob is correct here—we really do earnestly believe AI could kill all humans!” wrote Evan Hubinger. Hubinger is Anthropic’s head of alignment, which means he stress tests the company’s Claude models to ensure that they actually do what they’re supposed to do, and don’t develop harmful, deceptive, or dangerous behaviors as they get more capable—one of the things that those sounding the alarm are most worried about. “I personally think it is >10% within the next decade. I believe Anthropic is trying its best, but we do not yet have a plan to solve alignment for superintelligence and are not clearly on track to,” Hubinger continued. Geoffrey Irving, former Chief Scientist at the UK AI Security Institute, who also worked at Google DeepMind, OpenAI, and Google Brain, wrote, “I think we have a ~50% chance of all dying as a result of superintelligence, mostly due to actions in the next few to 10 years. I don’t expect to have that resolved to below 10% or above 90% before we either make it through, or we don’t.” Drake Thomas, who works on risk and safety for Anthropic, chimed in as well: “I would burn my equity to the ground in a heartbeat for a 1% higher chance we make it out of this situation alive.” And David Krueger, an AI professor with Mila, an artificial intelligence research institute based in Montreal, Canada who used to work at Google DeepMind, wrote, “I’m so SO sorry to people who are just learning that AI could kill us all.” Krueger also runs Evitable, a project that aims to get the word out to the public about the societal scale risks and harms of AI. The comments on X/Twitter are part of a much longer pattern of warnings from people working on AI development about the existential threats it poses to humanity, but Coxon, Hubinger, and Irving seem to be among the first to put the very short 10-year time frame on their apocalyptic forecasts. Geoffrey Hinton, who helped to pioneer the deep-learning techniques that underlie today’s AI, has been publicly warning about the humanity-ending risk AI poses since at least 2023, when he resigned from his role at Google so that he could speak more freely. That year, hundreds of AI experts and public figures signed a 22-word statement that read, “Mitigating the risk of extinction from AI should be a global priority alongside other societal-scale risks such as pandemics and nuclear war.” The signatories to that letter included Hinton and another so-called godfather of AI, Yoshua Bengio, as well as OpenAI CEO Sam Altman, Anthropic CEO Dario Amodei, and Google DeepMind CEO Demis Hassabis, among others. What Makes AI Extinction-Level Scary [Read Full Story on Nautil Us →](https://nautil.us/ai-engineers-are-having-their-oppenheimer-destroyer-of-worlds-moment-1284956/)