Jacob Coxon, a former researcher at OpenAI and Anthropic, said in an interview that staff at top AI labs genuinely consider the risk of humanity being wiped out by AI. According to him, leadership might soften their language in public, but in private conversations, the fear is real. The phrase 'We could all die' isn't just hyperbole in these discussions—it's treated as a real possibility.

The Biggest Threat: AI That Improves Itself

Coxon points to the riskiest moment: when AI can start upgrading itself. This could trigger an 'intelligence explosion'—models getting smarter at breakneck speed without human help, potentially surpassing us so much that we lose all control.

Risk Scenarios Inside AI Companies

In another interview, he added that teams inside these companies seriously discuss scenarios where AI could hack critical infrastructure or help create a deadly virus.

The AI Race Goes On, Despite the Fear

Coxon says developers are well aware of these risks, but the work doesn't stop. Anthropic, for example, worries about OpenAI and China, and doesn't want to be the first to slow down. The companies are basically trapped—each one is afraid a competitor will reach super-powerful AI first.