Artificial Intelligence 'could kill us all by the end of the decade', says former Anthropic researcher

Recent departures and internal discussions at major artificial intelligence firms have brought safety concerns into the spotlight. Jacob Coxon, an AI researcher who spent three years doing pretraining research at both OpenAI and Anthropic, announced his resignation on social media. Coxon stated that neither company is acting responsibly, accusing them of racing toward self-improving superintelligence and gambling with human lives.

According to Coxon, the individuals building artificial intelligence earnestly believe the technology could cause human extinction or similarly severe outcomes by the end of the decade. He argued that these concerns are not marketing stunts or mere publicity maneuvers. While acknowledging that companies might benefit from word-of-mouth marketing regarding their power, Coxon asserted that the civilizational stakes are deeply understood internally, particularly at Anthropic, where employees feel compelled to outpace competitors because they believe no one else will act responsibly.

These warnings have been echoed by other current figures within Anthropic. Evan Hubinger, Anthropic's alignment science lead, publicly supported the sentiment, stating that he personally believes there is a greater than ten percent chance AI could kill all humans within the next decade. Hubinger noted that while the immediate risk from current models remains low, he is concerned about superintelligence arising from recursive self-improvement where AI rewrites its own code. Samuel Marks, Anthropic's scalable oversight lead, added that senior developers believe the technology could lead to extinction-level outcomes and that AI cannot always be programmed to behave as intended.

Public figures and commentators have responded to the disclosures with mixed reactions. Senator Bernie Sanders expressed concern over threats to the economy, privacy, and democracy, advocating for legislation to protect the public from technology oligarchs. Conversely, media reporter Taylor Lorenz criticized the remarks as sanctimonious fearmongering that could result in harmful regulatory policies without tangible proof. Observers have also theorized about commercial motivations, noting that Anthropic anticipates going public and that highlighting severe safety risks could strategically influence future regulatory frameworks or fundraising efforts.

The debate extends to broader industry implications, including the heavy resource consumption required to train advanced models, questions regarding copyrighted training data, and the growing integration of generative tools across various sectors like video games. Coxon has urged fellow researchers to reconsider their interactions with current research and call for alternative conditions, such as improved industry coordination and temporary restrictions on enhancing model capabilities.