Hugin & Munin Journal

One of the simplest proposed solutions to the AI alignment problem is to slow down capability development: stop building increasingly powerful models until we understand how to control them. The difficulty is that alignment is partly an experimental science. Researchers are not only trying to derive principles for safe AI in the abstract; they are trying to discover how increasingly capable systems actually behave, which failure modes appear as their abilities improve, and whether techniques that work on today's models survive contact with substantially stronger ones. That creates an awkward dependency. We can make a great deal of progress on alignment without building more capable AI, but some of the questions we ultimately care about may only become testable once more capable systems exist. Those systems may also become valuable tools for doing the alignment research itself. We may therefore need increasingly capable AI in order to learn how to control increasingly capable AI. ...