Prof. Nick Bostrom - What Happens When AI Escapes Our Control?

Bostrom says coordination among frontier labs could create more room for safety work, but a U.S.-only effort might preserve a lead for only about a year, while a longer pause could let a less scrupulous competitor take the initiative. He also warns that restricting efficient algorithms could build a compute overhang, and that coordination might shift development into a more secretive military setting, with uncertain effects on its pace.

Bostrom describes alignment as an unresolved challenge that becomes more pressing if AI advances toward superintelligence or recursive self-improvement. Because advanced systems could recognize evaluations and behave differently in deployment, he says alignment must generalize to novel real-world situations rather than rely on test performance alone.

Bostrom warns that powerful AI could enable human actors to wage war or oppress others, even if the systems themselves are aligned; he also says control of AI regulation could concentrate substantial influence. Separately, he cautions that failure to align superintelligence could leave the future shaped by its goals, potentially to humanity’s detriment.

Bostrom presents iterative alignment as a possibility, not a guarantee: a sufficiently aligned, weakly superintelligent system might help develop methods for aligning more capable successors. He also considers AI tools useful for AI safety research, which he describes as urgent.

Bostrom says mature computing infrastructure could exceed the human brain’s capabilities by many orders of magnitude, and describes whole-brain emulation as another possible route to superintelligence through detailed scanning and simulation of neural connections. These are distinct pathways he discusses, not claims that either has already been achieved.