Jacob Coxon, who previously worked as a researcher at OpenAI and Anthropic training AI models, says he quit out of fear that artificial intelligence could wipe out humanity. According to Coxon, top AI labs are basically racing to build self-improving superintelligence. “The people building AI genuinely believe it could kill us all by the end of the decade,” Coxon said.

The Superintelligence Race

Coxon claims that Anthropic actually gets the risks better than most, but the push to develop more powerful systems hasn’t slowed down. The company worries that if they pump the brakes, a competitor could be the first to unleash a truly dangerous form of superintelligence.

What Anthropic Says About AI Risk

Anthropic AI safety specialist Evan Hubinger publicly backed Coxon. Hubinger puts the odds of AI wiping out humanity in the next decade at over 10%, and admits there’s still no real solution to the superintelligence safety problem.