The Alignment Catch-22: Do We Need More Powerful AI to Make AI Safe?

One of the simplest proposed solutions to the AI alignment problem is to slow down capability development: stop building increasingly powerful models until we understand how to control them. The difficulty is that alignment is partly an experimental science. Researchers are not only trying to derive principles for safe AI in the abstract; they are trying to discover how increasingly capable systems actually behave, which failure modes appear as their abilities improve, and whether techniques that work on today's models survive contact with substantially stronger ones.

That creates an awkward dependency. We can make a great deal of progress on alignment without building more capable AI, but some of the questions we ultimately care about may only become testable once more capable systems exist. Those systems may also become valuable tools for doing the alignment research itself. We may therefore need increasingly capable AI in order to learn how to control increasingly capable AI.

That is not an argument for automatic scaling. It changes the comparison that matters. Stronger AI will almost certainly make some alignment research easier; the question is whether our ability to understand and control AI improves faster than the capabilities we are creating.

Editorial image generated by the author.

How Much Can We Learn Without Stronger AI?

A halt to frontier scaling would not mean a halt to alignment research. Existing systems leave plenty to investigate: interpretability, adversarial testing, reward hacking, robustness, scalable oversight, AI control, monitoring, and the more basic problem of understanding what models represent and why they behave as they do. Nor do researchers need to wait for a genuinely dangerous system before studying dangerous behaviour. They can deliberately construct simplified forms of misalignment and ask whether their auditing methods detect them.

AuditBench, for example, contains 56 language models with deliberately implanted hidden behaviours, including forms of sycophancy, political preference and concealed loyalty that the models do not simply admit when questioned. These are not replicas of every future alignment failure. They are controlled test cases: known secrets against which an auditing method can be judged by whether it actually finds them.

A similar strategy can approximate the problem of supervising systems more capable than their supervisors. OpenAI's weak-to-strong generalization experiments used GPT-2-level models to provide supervision for GPT-4. Despite receiving imperfect labels from the weaker supervisor, the stronger model could recover a substantial share of the performance that weak supervision might otherwise have left inaccessible. The experiment was explicitly framed as a simplified analogue of a future problem in which humans may have to supervise systems whose answers they cannot reliably evaluate themselves.

The analogy has obvious limits. A weak language model is not a human supervisor, and GPT-4 in that experiment was not a strategically deceptive superintelligence. That does not make the experiment useless; it tells us what kind of evidence it provides. Existing models can expose weaknesses in proposed alignment techniques long before the full-strength version of the problem exists. A period of slower capability growth could therefore buy real scientific value by allowing researchers to probe current systems more thoroughly rather than treating every new generation as merely a stepping stone to the next.

The harder problem begins when those methods start producing diminishing returns.

Where Proxies Stop Being Enough

Suppose researchers become exceptionally good at detecting deceptive behaviour in today's models. That still leaves an empirical gap if future systems acquire strategic abilities today's systems lack. A model capable of recognising that it is undergoing a safety evaluation might behave differently during evaluation and deployment. A system able to plan coherently over weeks could expose weaknesses that never appear in short interactions. An AI substantially better than its supervisors at persuasion or vulnerability discovery might find ways around oversight that weaker systems cannot.

None of those possibilities has to be inevitable for the problem to matter. A safety technique can only be directly tested against behaviours the experimental system is capable of producing. Researchers can create artificial failure modes, construct adversarial scenarios and use weaker-versus-stronger models as proxies, but eventually a proxy may cease to answer the question we actually care about.

The logic resembles staged testing in medicine. A drug can be studied in cells and animals before anyone receives it. Those experiments can eliminate bad candidates, expose mechanisms and substantially reduce uncertainty, but they cannot establish every effect the drug will have in humans. At some point, further evidence requires entering the regime whose risks were previously being inferred indirectly.

AI alignment may have a similar research frontier. Below it, current systems and constructed proxies continue to generate useful evidence. Beyond it, additional confidence may require systems possessing abilities the earlier models simply did not have. The uncomfortable feature is that crossing this research frontier may also mean crossing into the territory we are trying to determine is safe.

When Stronger AI Helps — and Moves the Target

There is a second complication. AI is becoming not only the object of alignment research but one of its instruments. In 2026, researchers using automated alignment researchers had AI agents propose training methods, generate data, train models and iterate against safety benchmarks. Across ten measured categories of alignment failure, the strongest methods reduced the targeted failures and generalised to held-out evaluations and models up to 4.7 times larger than the target model.

These were deliberately bounded tasks. Reducing measurable deception, sycophancy or jailbreak behaviour on a benchmark is not the same thing as solving alignment. But the experiments demonstrate something important: parts of alignment research already consist of loops that AI can partially automate. Interpretability can involve analysing enormous quantities of internal model behaviour; adversarial evaluation benefits from producing more attacks than a human team could devise manually; experimental research repeatedly moves through hypothesis, intervention, measurement and revision. As models improve, they should become more useful inside those loops.

The optimistic version is a feedback process: better AI accelerates alignment research, better alignment makes the next generation safer, and the safer generation becomes an even better research tool. The weakness is that this story normally tracks only one side of the equation. The same model that becomes better at alignment research may also become better at autonomous action, software development, persuasion, strategic planning or scientific research.

Alignment can therefore improve dramatically in absolute terms while the safety margin shrinks. If our ability to monitor a system advances one step while its ability to find ways around that monitoring advances three, the one-step gain remains genuine but does not tell us that the situation has improved. The relevant quantity is the gap between capability and control.

Neither side is easy to measure. Capability is multidimensional: modest movement on a broad benchmark can coexist with much larger changes in coding, agentic operation or tool use. Control is harder still to quantify because some failures may remain invisible until the right environment or incentive appears. Yet the measurement problem does not remove the comparison. If capability repeatedly advances faster than our ability to understand, supervise and constrain it, AI-assisted alignment could become increasingly productive while still falling further behind. If AI becomes a much better alignment researcher faster than it becomes difficult to control, the opposite could happen. We do not know which curve is steeper.

Why Both Simple Answers Fail

A permanent version of “pause until alignment is solved” runs into one side of this problem. Imagine researchers spent ten years studying the strongest available systems without increasing their capabilities. Interpretability improved enormously, reward hacking became easier to detect, and oversight proved highly reliable within the regime being studied. The claim that ultimately matters would still concern something else: whether those techniques continue working when the model becomes substantially more capable than its supervisors. Complete empirical confidence about a regime we refuse ever to enter may simply be unavailable.

The inverse argument is no better. “We need stronger models to solve alignment” has no natural stopping point. We need the next model to learn how to align the next model; once it exists, the remaining uncertainty justifies another step. The same logic can be applied to AGI, and then again to something still more capable. At every stage the argument regenerates itself. “We need the next model for safety research” can become indistinguishable from “we need the next model.”

That problem becomes sharper because frontier AI is not developed like an isolated laboratory specimen. Systems that require enormous investments to train are also economically valuable, and the organisations building them have incentives to deploy them, commercialise them and use them to improve later systems. Nor is there a clean technological boundary between alignment and capability research. Making a model more reliable may make it safer and more commercially useful at the same time; tools capable of automating alignment experiments may also automate other AI research. Safety and capability development overlap precisely where we would most like to separate them.

Alignment-Gated Scaling

One possible response is staged development: extract as much alignment knowledge as reasonably possible from each capability level before deliberately moving to the next. Monitoring, oversight and control techniques would be attacked rather than merely demonstrated under favourable conditions, and evidence gathered at one level would help determine whether creating the next is justified. Call the principle alignment-gated scaling.

The gate is the important part. Frontier laboratories can conduct excellent safety research while still moving rapidly towards stronger systems. If poor safety evidence has no practical effect on whether the next training run happens, alignment can remain permanently one generation behind while appearing to advance quickly. A genuine gate accepts that stronger systems may eventually be needed for further alignment research without treating that fact as a licence to advance as quickly as possible. Elements of that logic are already appearing in frontier-lab governance: in September 2026, OpenAI wrote that alignment and monitoring were not sufficiently solved to continue responsibly scaling at maximum speed for much longer, while Anthropic’s Responsible Scaling Policy ties stronger safeguards and affirmative safety cases to capability thresholds.

The principle is much cleaner than any plausible implementation. We do not know exactly where dangerous capability thresholds lie, and particular abilities need not improve smoothly with aggregate model performance. Nor is there an obvious evidentiary standard for opening the gate. Safety cannot mean proving that failure is impossible; someone would have to decide what residual risk is acceptable and how uncertainty should be treated when the rewards for proceeding are large.

Then comes the coordination problem. A laboratory that waits to understand its current system may simply give competitors time to pass it. If frontier AI carries large economic or strategic rewards, voluntary restraint becomes difficult even for organisations that sincerely prefer a slower trajectory. Alignment-gated scaling therefore stops being only a technical principle and becomes an institutional one: can companies and governments construct gates that survive the incentives to ignore them?

The Experiment We Cannot Run Completely in Advance

The alignment problem is often presented as a choice between two clean strategies: solve safety first and then proceed, or keep developing AI because stronger systems will help solve the safety problems. Neither description fits particularly well.

We can probably learn much more from today's systems before we need tomorrow's. Model organisms, weak-to-strong supervision, interpretability, scalable oversight and AI control let researchers investigate future problems without immediately constructing their most dangerous possible form. Slower capability development could therefore buy genuine knowledge rather than merely postpone an inevitable transition.

But alignment is unlikely to become entirely independent of the systems it is trying to align. Some hypotheses may only become testable when the relevant capabilities exist, and increasingly capable AI may itself become one of the main tools used to test them. The target does not merely keep moving. Moving it may sometimes be necessary to discover whether our methods still work.

That leaves an unusually awkward form of technological experimentation. We may eventually reach a point where researchers have extracted most of the evidence available from one generation while the next decisive experiment requires creating a system whose safety is precisely what remains uncertain. There is no procedure that removes this dilemma; the best available strategy may be to make the steps smaller, learn more before taking each one, and demand stronger evidence as the consequences of being wrong increase.

Technically, the alignment problem may eventually require more advanced AI. Institutionally, the harder problem may be creating a system in which needing the next experiment does not automatically become an excuse for running it.