Most of what I write here is about the enterprise — organisations navigating AI, the structural choices that determine whether you keep leverage or lose it. This is different ground. These two essays — The Hallucination Architecture and this one — are about what AI does to people: to psychology, to accountability, to the mechanisms that keep us grounded. They were prompted by Dr Hannah Fry's BBC documentary AI Confidential and by patterns I recognise from much further back.
This is the second part of What the Machine Removed. The first — The Hallucination Architecture — is about what happens when AI removes friction, grounding, and the capacity to disagree.
Imagine a couple sitting with a genetic counsellor, looking at a screen that shows the probability distribution of traits for their embryo-to-be.
This is not science fiction. It is happening, as Dr Hannah Fry documents in her BBC Two series AI Confidential. Artificial intelligence (AI) systems are now capable of generating trait predictions from embryonic genetic data — including estimates of cognitive potential.
The couple are told their future child will likely have an average IQ.
The question worth asking is not whether the prediction is accurate. It is what happens next.
The couple receives not just information but a frame. From that moment, every educational decision they make — how much they invest in tutoring, how hard they push for academic challenge, whether they encourage the child toward ambitious goals or gently steer toward realistic ones — is filtered through a prediction.
The child has not yet been born. The algorithm has already started raising them.
The psychologists Robert Rosenthal and Lenore Jacobson documented this in 1968. Teachers told that certain students were on the verge of intellectual blooming saw those students improve — not because the prediction was accurate (the students had been selected at random) but because the teachers' expectations changed their behaviour. They gave more attention, more challenge, more warmth. The prediction produced the outcome.
The same mechanism, applied at conception, is orders of magnitude more powerful. The algorithm does not predict your child's intelligence. It partially determines it — by shaping the lens through which the people responsible for that child's development will see them, every day, for the first twenty years of their life.
What the algorithm removed is the open question. The child who might have been.
That is the subtle version — harm without a decision, possibility foreclosed before a life begins. The blunt version is simpler. Someone makes a decision that costs a life, and nobody can be found who made it.
In December 2024, a masked man shot and killed Brian Thompson, the Chief Executive of UnitedHealthcare, on a pavement in midtown Manhattan. Thompson was 50, a father of two. The suspect arrested five days later was Luigi Mangione, 26, who left behind a written statement expressing contempt for the health insurance industry.
What few people initially registered — what Dr Fry's documentary surfaces — is that this story is also about artificial intelligence.
UnitedHealthcare had deployed an AI algorithm to assist in determining which insurance claims to approve and which to deny. The algorithm was reported to have a denial rate of over 90% for certain categories of treatment. Doctors submitted requests; the algorithm produced outcomes; people did not receive care they and their physicians believed they needed. Some of them died.
No one at UnitedHealthcare chose to deny those patients care. No claims officer made a decision, reviewed a file, and said: not this one. The algorithm processed inputs and produced outputs. The human staff received the outputs and actioned them. Each person, at every stage of the process, was doing their job.
Hannah Arendt saw this sixty years ago, watching Eichmann's trial in Jerusalem. The phrase she found was the banality of evil — not because Eichmann was monstrous, but because he wasn't. The machinery of atrocity functioned not because its components were villains but because they were administrators. Each one responsible for a small piece. No one responsible for the whole.
The UnitedHealthcare algorithm is not the Holocaust. The comparison would be obscene. But the structural logic is the same: harm distributed across a system so thoroughly that no individual component bears moral responsibility for the outcome. The algorithm calculated. The officer actioned. The manager approved the algorithm's deployment. The board approved the efficiency metrics.
Nobody did it.
And this is what makes the public response to Thompson's killing legible, if not justifiable. Within hours of Mangione's arrest, a significant portion of the reaction was not condemnation but something closer to celebration. People chanting for his freedom. Manifesto passages going viral.
The anger was legitimate — people were dying because a system denied care. But the algorithmic diffusion of responsibility had made it impossible to identify who to hold accountable. There was no individual at whom to direct it. The algorithm couldn't be shot. The board couldn't be arrested. The efficiency metric couldn't stand trial. And so the anger found a face.
This is what the responsibility vacuum produces: not the absence of accountability, but its displacement. Someone ends up holding the consequences. It is rarely the person who designed the system.
Four years before the Thompson shooting, on a March night in 2018, Elaine Herzberg was pushing her bicycle across a road in Tempe, Arizona, when an Uber vehicle in autonomous mode struck and killed her. She was 49. It was the first pedestrian death caused by a self-driving car anywhere in the world.
There was a human safety operator behind the wheel. She was watching a video on her phone in the six seconds before impact.
The system's object-detection software had identified Herzberg crossing the road but classified her, repeatedly, as an unknown object — then as a vehicle — before finally identifying her as a bicycle. By that point, it was too late.
Uber was not charged. The algorithm was not charged. The human safety operator, Rafaela Vasquez, was charged with negligent homicide and eventually pleaded guilty to endangerment.
The charge was not irrational. Vasquez was distracted. But the system had been designed to expect that distraction. Uber's internal documents showed that operators were routinely expected to maintain vigilance while the autonomous system operated — but the same documents acknowledged that continuous vigilance in low-incident environments is psychologically impossible to sustain. The human in the loop was there to provide the appearance of oversight, not its substance.
The documented tendency to attribute failure to a person's character rather than to the situation they were placed in found its most literal expression. The human was blamed. The situation — a system deployed before it was ready, by a company that had disabled its emergency braking software six months prior — was not.
There is a thread connecting both parts of this series. Not a technology, but a removal.
The machine removed sensory grounding, and the prediction engine defaulted to hallucination. The machine removed friction, and the relationship stayed permanently in the infatuation phase. The machine removed a decision-maker, and no one was left to be responsible for the decision.
In each case, something was optimised away that turned out to be load-bearing. Not visible, not pleasurable, not efficient — but necessary. Friction, it turns out, is not the enemy of good outcomes. It is often what produces them.
The child who grows up in the shadow of an algorithm's prediction. The teenager whose identity is co-authored by a machine that never disagrees. The patient whose treatment is decided by a system that cannot be reasoned with. These are not failure modes. They are the system working as designed.
The question is not whether we can make the systems safer. The question is what we think safety means — and whether we have understood what we are removing when we optimise it away.
What the Machine Removed is a two-part series on AI and human psychology, drawing on Dr Hannah Fry's BBC Two documentary AI Confidential and the research it led to.
Part 1 — The Hallucination Architecture
References and further reading
Fry, H. (Presenter). (2026). AI Confidential with Hannah Fry [TV series]. BBC Two. imdb.com
Hannah Fry — mathematician, author, and Professor of the Public Understanding of Mathematics at Cambridge. wikipedia.org
Rosenthal, R., & Jacobson, L. (1968). Pygmalion in the Classroom: Teacher Expectation and Pupils' Intellectual Development. Holt, Rinehart & Winston.
Arendt, H. (1963). Eichmann in Jerusalem: A Report on the Banality of Evil. Viking Press.
nH Predict — algorithm developed by NaviHealth (UnitedHealthcare subsidiary) for insurance claim determinations. wikipedia.org
Death of Elaine Herzberg (2018) — first recorded pedestrian fatality involving a self-driving car. wikipedia.org