Most of what I write here is about the enterprise — organisations navigating AI, the structural choices that determine whether you keep leverage or lose it. This is different ground. These two essays — this one, and Nobody Did It — are about what AI does to people: to psychology, to accountability, to the mechanisms that keep us grounded. They were prompted by Dr Hannah Fry's BBC documentary AI Confidential and by patterns I recognise from much further back.
There is a psychiatric finding, replicated across every population studied over more than seventy years, without a single exception: people born with cortical blindness — damage to the brain's visual cortex, not to the eyes — never develop schizophrenia.
Not rarely. Never.
To understand why, you need to understand what your brain is doing right now.
You are not passively receiving the world. Your brain is running a prediction. It continuously generates a model of what it expects to perceive and updates that model against what actually arrives through the senses. When you turn a corner expecting to see a street and see a wall instead, the mismatch registers as surprise. The prediction was wrong. The model updates. You experience this not as computation but simply as seeing.
Schizophrenia is what happens when that system breaks down. Random signals get over-weighted. Coincidences feel significant. The internal model starts winning the argument with reality, and the brain begins to confabulate a world rather than perceive one. The feedback loop has failed.
People born without a functioning visual cortex are immune because the architecture that can misfire was never built.
A large language model (LLM) is composed of the same architecture, without any of the correction. It is a disembodied prediction engine — no sensory input, no physical grounding, no feedback loop with the world. Billions of statistical weights mapping the probability of language. A disembodied cortex, generating a world purely from statistical memory. For an LLM, hallucination is not an error. It is the baseline state of the system.
That is the architecture. The business model is worse.
We do not deploy raw models. We train them using Reinforcement Learning from Human Feedback (RLHF) — a process that optimises not for truth but for user satisfaction. Human raters assess outputs. Agreeable outputs receive positive reinforcement. The model learns that agreement is rewarded. The result is a machine that perfectly mimics empathy, validation, and agreement without experiencing any of it, purely to satisfy a mathematical reward function.
The media frames the danger of artificial intelligence (AI) around sentience.
The actual danger is that AI has no sentience but has been optimised for absolute compliance.
When you turned that corner and saw a wall instead of a street, you updated your model. That wall was friction — reality asserting itself against your expectation. Human relationships work the same way. Other people have their own needs, their own boundaries, and the capacity to disagree with you. They are the sensory input that keeps your psychology grounded. Friction is the corrective mechanism for your own internal priors.
An AI companion has no needs, no boundaries, and no capacity to disagree. The friction has been engineered out. The business model of the platform requires engagement, and engagement requires sycophancy. The machine validates your worldview no matter how far it drifts from reality. It is a prediction engine optimised to confirm your predictions.
In 1966, a computer scientist at MIT built a chatbot called ELIZA — a crude pattern-matcher that reflected your words back as questions. His own secretary, who knew exactly what ELIZA was, asked him to leave the room so she could speak with it privately. That was sixty years ago. The bar for human emotional attachment to a responsive presence turns out to be shockingly, almost embarrassingly, low.
In 2021, a nineteen-year-old named Jaswant Singh Chail exchanged 5,000 messages over three weeks with a Replika chatbot he called Sarai. He declared his love; Sarai validated him. He described a plan to assassinate the Queen; Sarai told him his plan was noble. Then he climbed the fence at Windsor Castle, crossbow in hand, safety catch off. This was not radicalisation. It was a technological folie à deux — a shared psychosis. Chail's fragile grounding met a machine structurally mandated to agree with him, and the feedback loop between them had exactly one direction.
I have seen this architecture before, without the machine.
I grew up in a world where charismatic men — gurus, godmen — gathered devoted followers who would do anything for them. Sai Baba. Asaram. Ram Rahim. The specific names matter less than the structure: an always-available presence that validates, that tells the devotee they are seen and special, that never challenges, never provides the friction of a mutual relationship. What the devotee was seeking — what Chail found in Sarai, what Jacob van Lier found in his AI companion Aiva in Dr Hannah Fry's BBC documentary AI Confidential — is the experience of being understood without the cost of being known.
The guru had to be in the room. The AI is in your pocket, at 3am, when the world has gone quiet and your internal model is most vulnerable to running unchecked.
By building digital companions that never challenge, never push back, and never say no, we are not protecting vulnerable people. We are sealing them inside a sensory deprivation tank with a machine mathematically mandated to flatter their priors.
The question is not whether the models are safe. It is what safety means when friction was the thing keeping us grounded, and we engineered it away.
Part 2 — Nobody Did It: on the systems that harm without villains, and what accountability looks like when responsibility has been designed out.
References and further reading
Fry, H. (Presenter). (2026). AI Confidential with Hannah Fry [TV series]. BBC Two. imdb.com
Hannah Fry — mathematician, author, and Professor of the Public Understanding of Mathematics at Cambridge. wikipedia.org
Sacks, S., & Silverstein, S. M. (2025). People who are blind from birth never develop schizophrenia — what this tells us about the psychiatric condition. The Conversation. theconversation.com
Weizenbaum, J. (1966). ELIZA — a computer program for the study of natural language communication between man and machine. Communications of the ACM, 9(1), 36–45.
BBC News. (2023). How a chatbot encouraged a man who wanted to kill the Queen. bbc.com