Hugin & Munin Journal

Reading Eliezer Yudkowsky’s Inadequate Equilibria sent me back to a familiar problem: systems in which many participants can recognise a better outcome, yet no one can reach it safely alone. This is often where the language of Nash equilibrium enters the discussion. It is a useful concept, but only if it remains narrower than our ordinary idea of a system being “stuck”. In John Nash’s original formulation , an equilibrium is a combination of strategies in which no player can improve their own payoff by changing strategy alone while everyone else’s strategy remains fixed. The definition says nothing about whether the outcome is fair, efficient, desirable or unique. Nor does it require the world to stop changing. It identifies a form of strategic stability: given what everyone else is doing, unilateral deviation does not pay. That distinction explains how individually defensible choices can assemble themselves into a collectively poor result. The obstacle may not be ignorance, malic...

Some books are valuable because they introduce an unfamiliar idea. Others give a precise shape to something you had already noticed without knowing how to organise it. Eliezer Yudkowsky’s Inadequate Equilibria: Where and How Civilizations Get Stuck belongs firmly in the second category for me. The book is often described as an explanation of why institutions remain trapped in bad arrangements. That is true, but it leaves out the more difficult question Yudkowsky places in the reader’s hands: when should you trust that an existing system has already found the obvious improvements, and when might it be reasonable to think that you can see something it has missed? This is not permission to assume that experts are fools or that private conviction outranks accumulated knowledge. It is an attempt to distinguish justified deference from deference performed automatically. The distinction depends less on how intelligent or prestigious the people inside a system appear than on what the syst...