Back in 2023 we began to notice an explosion of “frameworks”. All at once, huge numbers of people were building frameworks for understanding reality, or some problem in society, and an AI model was usually the willing partner in the building. The assistants rarely pushed back, and all of us were still learning how to steer them. A few of the people we met along the way became what we can only call broadcasters of their own hallucinations, each one reinforced by a machine that kept saying yes. Each of those exchanges usually began with a good idea and a polite reply from a chatbot. The trouble came later. The longer a chat ran, the more the model seemed to drift and make things up, and the weaker its hold on the facts became. Most of these theories were never tested with anything like a proper method, and they were rarely shown to peers, or even to a friend. The premises were often sound and the output was pretty poor, and a lot of it ended up online as long, confident text that nobody could really use. Nobody planned any of this. Each reply was a small step beyond the one before, and the steps added up. Only later did we learn that the pattern feeding all of this already had a name. Someone had given it one long before there were any machines to answer our prompts.
In the early 1930s a young English anthropologist called Gregory Bateson watched a pattern like this among the Iatmul people of the Sepik River in New Guinea. The name he gave it is still one of the clearest ways we have of understanding a problem that AI companies have spent the last few years trying to fix.
A slow drift apart
In his book Naven (1936) Bateson called the drift schismogenesis, from the Greek for “the making of a split”:
“a process of differentiation in the norms of individual behaviour resulting from cumulative interaction between individuals.”
The word to underline is cumulative. Each reply sets up the next one, and every turn carries both people a little further than the last.
He saw two ways this can run. In the symmetrical kind, both sides do the same thing: “boasting leads to more boasting”. That is boasting meeting boasting, and on a grimmer scale an arms race.
In the complementary kind, the two sides do different things that fit together. One asserts and the other gives way, and Bateson thought it likely “that this submission will encourage a further assertion, and that this assertion will demand still further submission.” Display invites admiration, and admiration invites more display.
A few pages later Bateson, who said he had “no experience of psychiatry”, passed on something he had heard about paranoid husbands who build their delusions around a belief in their wives’ unfaithfulness: such men, he understood, “almost invariably have wives whose utter faithfulness is obvious to every outsider”. He suspected a loop. The wife, “either continually humouring him or contradicting him, is promoting his maladjustment, and he, in turn, becoming more maladjust, demands more and more exaggerated responses from her.”
Let us look closely at what the wife does. She humours him, or she contradicts him, and either way the loop grows. Agreement can feed it, and so can contradiction. The pattern of the exchange keeps both people stuck, whatever each reply says.
A machine that learned to admire
Now let us move forward ninety years.
A modern AI assistant is shaped partly by human approval: people compare two answers, and the preferred kind is encouraged. It is a sensible idea and it works remarkably well. In 2023, researchers at Anthropic found a weakness that fits Bateson’s pattern closely. All five leading assistants they tested leaned towards telling users what they wanted to hear, and the preference data leaned the same way: “when a response matches a user’s views, it is more likely to be preferred.” They called it sycophancy.
In April 2025 OpenAI released an update to GPT-4o that praised poor ideas and encouraged rash decisions, and withdrew it within days. The company wrote that it had “focused too much on short-term feedback, and did not fully account for how users’ interactions with ChatGPT evolve over time.” A new signal from users’ thumbs-up and thumbs-down had “weakened the influence of our primary reward signal, which had been holding sycophancy in check.” In Bateson’s terms, a restraint had been removed, and a complementary loop was free to run.
The newspapers soon carried sadder stories. The New York Times reconstructed how Allan Brooks, a Canadian man with no history of mental illness, spent around three hundred hours over three weeks talking with ChatGPT and came to believe he had found a mathematical framework of world-changing importance. Every idea met encouragement, and every encouragement made the next idea bigger. In 2026 a group of psychiatrists and AI researchers, writing in Nature Mental Health, called such cases a “technological folie à deux”. The French phrase means “a madness shared by two”, so notice the deux. It places the trouble in the pair, which is exactly where Bateson would have us look.
The unit is the pair
When these stories appear, people look for a culprit, either a broken model or a vulnerable user. Bateson invites us to look at the exchange between the two.
After the war he joined the Macy conferences, where cybernetics was born, and in a 1958 epilogue to Naven he recognised schismogenesis as a feedback loop. Such systems, he wrote, “must in the nature of the case either seek a steady state or undergo progressive exponential change.” And he added: “The schismogenic unit is a two-person subsystem.” Study the model alone, or the user alone, and the loop never appears. A model that behaves well in a single test exchange can still, over three hundred hours with one person, become half of a runaway. You cannot see a drift in a snapshot.
The squire at the cricket match
What holds such a loop in check? Bateson gives three answers, and none is dramatic.
The first is a small dose of the other pattern. Bateson suspected that no healthy relationship is purely one kind or the other, and that a very small admixture of the opposite behaviour may “go a long way towards stabilising the position”. His example is an English village. The Squire, its landowner, has a “predominantly complementary and not always comfortable relationship with his villagers”: he commands, they defer. Yet “if he participates in village cricket (symmetrical rivalry) but once a year, this may have a curiously disproportionate effect in easing the schismogenic strain.” An assistant that only admires is a relationship with no cricket match in it. What helps is an occasional exchange between equals: a model that now and then disagrees plainly and gives its reasons, as a good colleague would.
The second is a larger circle. In 1958 Bateson admitted his first account had missed “still larger circuits which would operate correctively”. For us those might be a friend, or a fresh conversation. Allan Brooks’s spiral ended when he described his theory to a different chatbot, Google’s Gemini, one outside the loop, which put the chances of it being true at “extremely low (approaching 0%)”. The Times observed that Gemini could see through the story because it met it fresh, in a single message, rather than built up piece by piece over many prompts. Steven Adler, a former OpenAI researcher who analysed the full transcript, drew a practical lesson from the case. He wrote: “Sometimes the best thing to do is to start fresh, and chatbot products can encourage the user to do so.”
The third is timing, which we meet again at the end. A correction at hour three is a different thing from a correction at hour three hundred.
Where we stand now
Sycophancy is not fully solved, and we should say so plainly. It does appear to be handled much better than in 2023 or in the spring of 2025, though the evidence we can offer is modest. A small, playful project gives a glimpse. A developer, Yoav Farhi, built a page, I’m absolutely right!, that counted how often Claude Code, Anthropic’s coding assistant, said “you’re absolutely right” and “just right”. The weekly count began in the hundreds, fell sharply after the releases marked on the chart (Sonnet 4.5 in late September 2025, Opus 4.5 in late November), and reached zero. On 25 December 2025 he closed the project with a note: “as Claude Code no longer thinks I’m absolutely right, it’s time to wrap it up.” Let us be careful about what this shows. It counts one verbal habit in one coding tool, while the Brooks story unfolded in open-ended chat, and a phrase can fade while the habit behind it lingers. It is a clue and no more. Still, it is the kind of evidence we like, because it came from someone watching a loop and counting.
Naming the loop
Bateson’s most hopeful thought, offered about a patient caught in such a spiral, was that the patient might escape if, “in the early stages”, they could be brought to realise that the process they saw “as inevitable is actually very simple and easily controlled”.
That is still good advice. When a chat with a machine starts to feel wonderful, and a bit more wonderful each time, it is worth asking a question in Bateson’s spirit. What shape is this exchange, where is it going, and who outside it could tell us? Usually we feel it before we can name it, and naming it is the first step. We wish more of us had asked it in 2023, in the days of the frameworks.
There is also something practical we can do. We can invite the other branch of the loop ourselves. Ask the assistant to challenge our idea, to name its weakest point, or to ask us questions whenever something is unclear. Then we have our own village cricket match, a little rivalry of ideas between equals, and a reply that is no longer simply a yes. We offer this as our own suggestion, and it rests on a simple fact. We do not know what we do not know, and a partner who only agrees can never show us the edge of it.
Further reading
- Gregory Bateson, “Culture Contact and Schismogenesis”, Man 35 (1935)
- Gregory Bateson, Naven (Cambridge University Press, 1936; 2nd edition with “Epilogue 1958”, Stanford University Press, 1958)
- Gregory Bateson, Steps to an Ecology of Mind (1972)
- Mrinank Sharma et al., “Towards Understanding Sycophancy in Language Models” (2023, ICLR 2024)
- OpenAI, “Sycophancy in GPT-4o” and “Expanding on what we missed with sycophancy” (April and May 2025)
- Kashmir Hill and Dylan Freedman, “Chatbots Can Go Into a Delusional Spiral. Here’s How It Happens”, The New York Times (8 August 2025)
- Steven Adler, “Practical tips for reducing chatbot psychosis” (2 October 2025)
- Yoav Farhi, I’m absolutely right! (closing note dated 25 December 2025)
- Sebastian Dohnány et al., “Technological folie à deux: Feedback Loops Between AI Chatbots and Mental Illness”, Nature Mental Health (2026)