Anthropic researcher flags double-digit AI extinction risk

An Anthropic safety researcher has said there is more than a 10% chance that artificial intelligence could kill all humans. The warning came the same day a former colleague said he was leaving the company because he believed AI labs were treating human safety too lightly.

Jacob Coxon said he was resigning from Anthropic and argued that both Anthropic and OpenAI are not behaving responsibly. He said the industry is pushing toward self-improving superintelligence while gambling with human lives. The remarks add to a widening internal debate over how quickly frontier AI systems are advancing and whether safeguards are keeping pace.

Internal alarm inside AI labs

Self-improvement in this context refers to AI systems becoming better at improving themselves with limited human involvement. Coxon said that process is not fully possible yet, but he believes major labs are working toward it. He warned that the technology should not be underestimated because it could become extremely capable at hacking, industrial disruption and gathering resources.

Coxon also said some people working on AI believe the technology could become capable of killing humanity before the end of the decade. That is a stark assessment from inside one of the sector’s best-known companies. It also comes as leading AI groups continue to attract large amounts of capital and move closer to public market debuts.

Market scrutiny deepens around AI safety

The comments are likely to intensify scrutiny of the safety culture at companies building advanced AI models. They highlight a tension that has been present for much of the current AI boom: investors and customers are rewarding rapid technical progress, while some researchers argue that the risks are moving faster than the controls meant to manage them.

Evan Hubinger, an alignment science lead at Anthropic, responded that Coxon was right and said the company has no plan for the scenario he described. His response suggests that concern is not limited to one departing employee. It points to a broader unease inside the field about whether current safety strategies are adequate for systems that could become far more powerful.

The episode matters for markets because Anthropic and OpenAI are among the companies most closely watched in the race to build frontier AI. Public confidence, regulatory pressure and internal staff morale all shape how those companies are valued and how quickly they can scale. When researchers publicly question whether the industry is prepared for catastrophic outcomes, the debate shifts from technical capability to governance and control.

This article is not investment advice and recommends no asset, level or direction; a single session's move is not evidence of a trend. For background see Stocks, Bear Market, BIST 100, and for terms the finance glossary.