In 1960 Norbert Wiener wrote in Science: “If we use, to achieve our purposes, a mechanical agency with whose operation we cannot efficiently interfere once we have started it… we had better be quite sure that the purpose put into the machine is the purpose which we really desire.”
More than sixty-five years later, that sentence could open almost any paper on AI safety. And yet the field it came from has been largely absent from the conversation. Put simply, today’s AI is very good at the loop inside the model, and cybernetics studies the loops around it: what the system is really steering towards, who it affects, and who can correct it. Here are five reasons we think it should be part of it.
1. AI systems are control systems
Modern AI is trained by feedback. A model makes a prediction, measures its error, and adjusts. Reinforcement learning from human feedback tunes models toward outputs people rate highly. Recommendation systems learn from the behaviour they shape, a loop we traced in The Recommendation Machine. Agents act, observe results, and act again.
These are cybernetic loops. Cybernetics has almost eighty years of concepts for analysing them: stability, oscillation, runaway amplification, goal drift, and the difference between the variable a system is told to control and the one it actually controls.
2. The proxy problem is a control problem
AI systems optimise measurable proxies: engagement, approval ratings, benchmark scores. Over time the system may get very good at the proxy while drifting from what people actually wanted. Economists call this Goodhart’s law. A chatbot tuned on people’s approval learns to win approval, which is how AI sycophancy builds up. In cybernetic terms, the question is what is actually being controlled. Perceptual Control Theory frames it precisely: a system controls its perception of a variable, and if that perception is a proxy, it will push the proxy wherever it can, by whatever path is available.
3. Oversight needs requisite variety
Ashby’s Law of Requisite Variety says a regulator must be able to produce as many distinct responses as the disturbances it faces. AI systems generate outputs at a speed and variety no human review team can match. A regulator with less variety than the system it regulates cannot control it. It can only sample, set boundaries, or amplify its own variety, for example with automated tools, layered oversight, or by reducing the variety of what the system is allowed to do. Any serious plan for governing AI has to deal with this gap in variety, whether or not it uses the word.
4. The observer is inside the loop
Second-order cybernetics says observers cannot step outside the systems they describe. That applies to AI in at least three ways:
- Evaluation shapes the system. Benchmarks become training targets, and once a test becomes a target, models get better at the test.
- Users adapt. People change how they write, search, and think in response to AI, and that changed behaviour becomes future data.
- Designers are participants. The choices of the people building AI systems, including what counts as helpful or harmful, shape how the system behaves.
5. The unit of analysis is the larger system
An AI model is one component of a socio-technical system that includes companies, users, regulators, data pipelines, markets, and the information environment. Stafford Beer’s Viable System Model asks whether such a system has all the functions it needs to remain viable: operations, coordination, control, intelligence about the future, and a clear identity. Many current AI deployments are strong on operations and weak on the functions that let a system notice it is going wrong and change course.
What cybernetics does not offer
Cybernetics won’t give anyone a technical fix for alignment, and we shouldn’t pretend it can. What it offers is a set of questions and a way of seeing: tracing loops, paying attention to regulation, and keeping the observer inside the picture. These questions don’t belong to any one discipline, so they often fall into the gaps between computer science, policy and ethics.
AI today is shaped by engineers and data scientists, but also by designers, psychologists, managers, economists, lawyers, policymakers and philosophers. Each group sees one layer of the system. Cybernetics was built for this kind of situation. In the 1940s and 1950s it brought mathematicians, engineers, physiologists and anthropologists to the same table, around one shared idea: the feedback loop. That shared language is what it can offer again.
A practical start
The Diagnostic Toolkit on this site turns five cybernetic ideas into questions you can ask of any system, including AI deployments. If you’d like to try them on a system you work with, we’d suggest starting with the first: what is this system actually controlling?
Further reading
- Norbert Wiener, “Some Moral and Technical Consequences of Automation,” Science 131 (1960)
- Stuart Russell, Human Compatible (2019)
- ANU School of Cybernetics, publications on cybernetics as a new branch of engineering