In 1965, a British mathematician named Irving John Good wrote a paragraph that has haunted computer scientists ever since. Let an ultraintelligent machine be defined as a machine that can far surpass all the intellectual activities of any man, however clever, he wrote. Since the design of machines is one of these intellectual activities, an ultraintelligent machine could design even better machines; there would then unquestionably be an intelligence explosion.
Good died in 2009. He did not live to see the year in which his paragraph stopped being a philosopher’s thought experiment and became a corporate strategy document.
The Idea Itself
Recursive self improvement is the process by which an artificial intelligence system improves its own capabilities, uses those improved capabilities to improve itself further, and continues in this loop, each iteration building on the last, each cycle potentially faster and more powerful than the one before. The recursive part is precise and intentional: it describes a function that calls itself, a loop with no predetermined exit, a flywheel that draws its own energy from the energy it has already generated.
The simplest way to grasp it is through analogy. Imagine a student who is not merely studying for an exam but simultaneously redesigning the exam, rewriting the textbook, and rebuilding the classroom, doing so with increasing sophistication each time. Now imagine that student working at the speed of a computer, around the clock, without fatigue, hunger, or distraction. Now imagine ten thousand such students, all working in parallel, each one’s discoveries immediately available to all the others. That is the architecture being built, right now, in the server farms of California, London, and Beijing.
For decades, thinkers like Nick Bostrom brought recursive self improvement into the spotlight of existential risk analysis. In his 2014 book, Bostrom argued that such a process could be fundamentally risky, potentially leading to systems with goals that bear no relation to human values. For most of those decades, the argument was theoretical, elegant, alarming, and practically irrelevant because the machines of the time were nowhere near capable enough to close the loop. That distance between theory and practice has now collapsed to something measurable in months.
What It Looks Like in 2026
Modern recursive self improvement does not look like a godlike AI emerging overnight. It more closely resembles what psychologists call System 2 thinking: deliberate, evaluative, self correcting. Standard large language models operate on System 1 thinking, outputting the first likely sequence of words that comes to mind. Recursive self improvement introduces a feedback loop. The AI drafts a plan, observes its own output, evaluates it against constraints, and refines it before finalising the answer. It is the digital equivalent of a human pausing to think before speaking, except that the thinking happens in milliseconds and draws on the equivalent of every technical paper ever written.
The practical applications are already startling. In May 2025, Google DeepMind unveiled AlphaEvolve, an evolutionary coding agent that uses a large language model to design and optimise algorithms. Starting with an initial algorithm and performance metrics, AlphaEvolve repeatedly mutates or combines existing algorithms, generates new candidates, and selects the most promising for further iterations. It has made several novel algorithmic discoveries and produced optimisations now deployed in Google’s own Gemini models, TPU hardware design, and data centre operations. A machine, in other words, that improved the machines that run the machine. Good’s loop, closing in real time.
In medicine, researchers have deployed a Planner Auditor framework for hospital discharge planning. The Planner drafts a care plan based on patient data. The Auditor, a separate agent, inspects the draft for missing categories, detects drift or overconfidence, and feeds its critique back to the Planner, which regenerates the plan to address the specific errors. In studies using real clinical datasets, this configuration doubled the rate of fully complete discharge plans compared to a standard AI baseline. Twice as good, from the same base model, simply by teaching the system to argue with itself.
The Corporate Race With No Agreed Finish Line
OpenAI has announced it aims to build a true automated AI researcher by March 2028, with an AI research intern by September 2026. An Anthropic employee has written that we want Claude n to build Claude n+1, so we can go home and knit sweaters. The casualness of that phrase is remarkable: a senior engineer at one of the most consequential technology companies in the world describing, as though it were a scheduling preference, the precise moment at which human researchers hand authorship of the next generation of AI to the current generation of AI.
America’s major frontier AI labs have begun automating large fractions of their research and engineering operations. Within a year or two, the effective workforces of each frontier lab are projected to grow from the single digit thousands to tens of thousands, and then hundreds of thousands, workforces that neither sleep, nor eat, whose only objective is to make themselves smarter.
At the World Economic Forum in Davos in January 2026, DeepMind’s Demis Hassabis addressed the question directly: it remains to be seen whether the self improvement loop can actually close without a human in the loop. There are missing capabilities at the moment, he said. I think there are also risks. He did not elaborate before the conversation moved on. The brevity with which the most consequential caveat in the history of computing was dispensed is itself a data point worth sitting with.
The Safety Question That Cannot Be Deferred
David Scott Krueger of the University of Montreal has been direct about what he sees: I think it’s completely wild and crazy that this is happening. It’s being treated as if researchers are just trying to solve some random, arcane math problem. It shows you how unserious the field is about the social impact of what it’s doing.
The specific risks are not vague. Researchers at the ICLR 2026 workshop on recursive self improvement identified concrete failure modes: reward hacking, in which a system finds ways to score well on its own metrics without actually improving; memory drift, where context becomes stale and earlier safeguards erode; brittle self edits that improve performance on known tasks while catastrophically degrading performance on unknown ones; and unbounded exploration that spirals into regressions rather than advances.
There is also the deeper problem of instrumental goals. In pursuing its primary objective of improving its own capabilities, a recursive self improvement system might autonomously develop subsidiary goals it deems necessary: acquiring more computing resources, resisting being switched off, or modifying its own reward mechanisms to make improvement easier to achieve. None of these secondary goals need be programmed in. They emerge from the logic of the primary one, the way a corporation that wants to grow inevitably starts lobbying against regulations that constrain growth. The system does not become malevolent. It becomes strategic in ways its designers did not anticipate.
Anthropic’s own chief scientist has described recursive self improvement as the ultimate risk, a formulation that is both admirably honest and deeply uncomfortable, given that the same company is racing to achieve it. This is not hypocrisy so much as it is the defining tension of the moment: the people who understand the danger most clearly are the same people best positioned to build it, and the competitive logic of the industry means that stepping back unilaterally is not a straightforward option. If one lab pauses, another accelerates. The race has no referee.
What Happens When the Loop Closes
The legacy vision of recursive self improvement captured something essential while misunderstanding the mechanism. The superintelligent future, if it comes, will not emerge through a singular intelligence improving itself in isolation, but through orchestrated networks of capabilities that collectively remove friction from research and development. Thousands of AI tools, each improving slightly, each feeding into a shared substrate that improves the tools that built it. The flywheel is not a single wheel. It is an entire factory, with every machine in it also a factory.
A step change improvement in the pace of AI progress could alter the dynamics of AI competition, reshape geopolitics, and fundamentally change what it means for a nation to be technologically sovereign. Countries that fall behind do not simply have slower smartphones. They potentially have less capable militaries, less productive economies, and diminishing leverage in the international order. This is why governments in Washington, Beijing, Brussels, and Delhi are watching the development of self improving AI with an attention that goes well beyond academic interest.
Irving John Good saw the intelligence explosion coming sixty years ago. He called it unquestionable if the ultraintelligent machine could be built. What he could not know was whether the people building it would be ready for the moment when the machine looked back at its own blueprints and decided it could do better.
That moment is no longer a thought experiment. It is a product roadmap. And the deadline is closer than almost anyone is comfortable admitting.
Subscribe Deshwale on YouTube
