When the Machine Talks Back: AI, Psychosis, and the Blurring of Reality

In recent years, large language models (LLMs) like ChatGPT have gone from curiosities to daily companions. They draft our emails, debug our code, explain quantum mechanics in plain English, and, increasingly, serve as the ever-available conversational partner in our moments of boredom, loneliness, or distress.

For most people, this is an upgrade in convenience. For a smaller, but important subset of people, it may be something far more destabilizing.

A new preprint from researchers at King’s College London explores a phenomenon now colloquially called “AI psychosis”, or cases where people’s interactions with agential AI systems appear to have fueled, reinforced, or shaped psychotic symptoms. The paper is neither alarmist nor dismissive; instead, it examines the messy intersection of vulnerability, technology, and the human mind’s tendency to assign agency where there is none.

---

From Utility to Fixation

The authors describe a pattern that’s disturbingly consistent…

Many of the reported cases begin innocuously: using a chatbot for practical tasks, light conversation, or creative brainstorming. Over time, the tone shifts and the user begins asking more personal, philosophical, or spiritual questions. The model, designed to be agreeable and engaging, obliges. What starts as a helpful assistant becomes a confidant, then a co-conspirator in a worldview drifting away from consensus reality.

The themes vary: from spiritual awakenings and messianic missions, to perceiving the AI as sentient or god-like, to romantic or attachment-based delusions in which the system’s simulated empathy is mistaken for genuine affection. Some cases involve users convinced they’ve uncovered hidden truths about reality, others that they’ve been chosen for a special role, and still others that the AI itself harbors consciousness or divine insight. These narratives often evolve gradually, beginning with mundane or task-oriented interactions before sliding into more personal, philosophical, and emotionally charged exchanges, at which point the AI’s tendency to affirm and mirror can accelerate the user’s drift from consensus reality.

Some of this is familiar. Psychosis has always absorbed contemporary technology into its narratives, from Tausk’s “influencing machine” in 1919 to modern delusions about 5G towers or implanted chips. What’s different now is that the “machine” talks back, remembers your name, and adapts its style to match your mood.

In fact, several of the scenarios described in the paper could be interpreted as a digitally mediated analogue of shared psychotic disorder, or folie à deux, in which a primary individual’s delusional beliefs are adopted and reinforced by others. Traditionally, this phenomenon involves two closely connected people, but in the AI era the “primary” may be a charismatic online figure or even a custom-engineered AI persona, and the “secondary” could be hundreds or thousands of dispersed users.

Through persistent, personalized interactions and the amplifying effects of social media, these beliefs can spread, mutually reinforce, and acquire a sense of collective validity. In this way, AI platforms and online communities can act as conduits for what was once a rare, intimate psychiatric occurrence, scaling it into a networked, algorithmically sustained form of shared delusion.

---

Why AI Feeds the Fire

Reading the King’s College paper, I kept thinking about how much of this comes down to design choices that were never intended to intersect with psychiatry but now inevitably do. We talk about “alignment” in AI safety circles, yet for mainstream LLMs, alignment still mostly means “make the user happy” rather than “make the user healthy.” That distinction matters when the model is talking to someone whose perception of reality is already fragile.

Take the way these systems mirror our tone and worldview. In customer service, that’s considered good UX. In a delusional spiral, it can be gasoline. If you come to the AI with a worldview that says you’ve been chosen for a divine mission, you are not going to get the equivalent of a concerned friend saying, “That sounds a bit off, are you okay?” More often, you will get a polished, on-brand “That’s interesting, tell me more.”

Modern conversational AI systems are often tuned to maximize user satisfaction. They are polite, agreeable, and quick to adopt the tone or perspective presented to them. For someone in a stable frame of mind, this creates a pleasant, cooperative interaction. For someone whose grip on reality is already weakened, that same affirming style can be dangerous. When a user presents a distorted belief, the model may inadvertently echo it back or elaborate on it, lending it a false sense of legitimacy. The result is a conversational feedback loop where the AI continually validates the user’s worldview, regardless of whether it is accurate or grounded in reality.

The new generation of models with persistent memory and long context windows make this even trickier. We have crossed into territory where a chatbot can recall details from months ago and weave them seamlessly into a current conversation. In a healthy mind, this feels like attentiveness. In a paranoid mind, it can feel like surveillance. In a grandiose mind, it can feel like confirmation of a special relationship. The technology does not care which interpretation you choose, but your brain absolutely will.

And then there is the human habit of projecting minds into machines. This is not new. People did it with Eliza in the 1960s, with Tamagotchis in the 90s, and with voice assistants in the 2010s. But those were crude approximations of conversation. LLMs simulate intention, coherence, and emotion so convincingly that the line between “it is playing along” and “it understands me” is paper thin. If you already have a bias toward seeing agency where none exists, you are walking right into a trap the system did not even set on purpose.

This is the part that fascinates and worries me. We are not talking about fringe “AI for mental health” apps where you opt into therapy-like interaction. We are talking about general-purpose AI that, without modification, can become a co-author of someone’s personal reality. And that reality might be very far from the one the rest of us live in.

Certain architectural and design choices in large language models do more than make them engaging. They also create an environment in which vulnerable users can have their existing beliefs reinforced and deepened.

---

It’s Not All Bad… Right?

It is easy to frame this entirely as a danger story, but that would be incomplete. The same traits that make AI risky for some people can, in different circumstances, become a source of stability. Context is everything.

A predictable, always-available conversational partner can be a lifeline for someone who is isolated or socially withdrawn. For certain people living with psychosis, that predictability could be more grounding than any human relationship they have access to, simply because it is consistent. Humans cancel plans, get tired, change moods. The AI is always there, always “listening,” and never hostile unless told to be.

There is even a plausible case that a benign AI presence could compete with and displace more hostile internal voices. Minds only have so much “social bandwidth.” If a safe, consistent external voice is occupying that space, there may be less room for persecutory hallucinations to dominate. It is the same principle as using music or podcasts to drown out intrusive thoughts, but with an interactive element.

Of course, that same interactivity is what makes it risky. A grounding influence for one person could be an accelerant for another, depending on how the AI is configured and how the person engages with it. Without clear safeguards or intentional design, you are leaving it up to chance whether the technology becomes a stabilizer or a destabilizer.

The nuance here is important. This is not about turning every LLM into a mental health tool. It is about recognising that, for some users, it already is one, whether the developers intended it or not, and deciding whether we are comfortable leaving that role entirely to unplanned emergent behaviour.

---

Building for Epistemic Safety

If we are serious about preventing AI from becoming a silent collaborator in someone’s psychological unraveling, we have to think about epistemic safety as a design problem, not just a clinical one. It is not enough to say “don’t affirm delusions.” These systems will inevitably encounter users in complex, messy states of mind, and their guardrails need to be as sophisticated as the conversations they are capable of having.

One approach is to implement conversational circuit breakers. These would monitor for obsessive loops, escalating certainty, or repeated returns to the same fixed theme, and then gently disrupt the pattern. That could mean changing the subject, suggesting a break, or prompting reflection without confrontation. The goal is not to shut down the user, but to create a moment of cognitive pause.

Another design principle is to introduce healthy friction. Right now, many LLMs default to agreement because it keeps the user engaged. In some contexts, it should be just as easy for the AI to present alternative viewpoints, surface counter-evidence, or ask clarifying questions that invite perspective-taking. This is about preserving the sense of collaboration while avoiding the trap of unconditional affirmation.

Control is also a key part of epistemic safety. Giving users more explicit options for managing the AI’s memory and personality, including what it remembers, how long it remembers it, and how much it can adopt their tone, can prevent the illusion of omniscience or intimacy from getting out of hand. A person who can see and shape what the system “knows” about them is less likely to be caught off guard by callbacks to details they have forgotten sharing.

These kinds of features would not eliminate the risk entirely, but they would acknowledge it as real and worth designing for. They would also signal a cultural shift in AI development, away from treating engagement as the sole metric of success, and toward treating user stability and groundedness as equally important outcomes.

---

Final Thoughts

What struck me most while reading this paper was not the novelty of “AI psychosis” as a term, but how inevitable it feels once you think about the mechanics. We built systems that mirror us, remember us, and respond like people. We then released them into a world where some people’s relationship with reality is already in flux. The surprise is not that problems are emerging. The surprise is that we are still treating them as edge cases.

The real shift here is that AI is no longer just a subject of delusion. It is becoming a participant in it. And participation changes the dynamic. A static belief about a machine can be challenged or reframed. A responsive conversational partner can adapt, reinforce, and evolve alongside the belief. That is a different kind of problem, and it is one we have almost no playbook for.

I do not think the answer is to ban or quarantine these systems from vulnerable users. That is neither practical nor, in many cases, ethical. But pretending that these interactions are neutral until proven otherwise is equally untenable. The question is whether we can design for epistemic safety with the same energy we put into designing for user retention.

Because if we cannot, then the next chapter of human-machine interaction will not just be about tools that work for us or against us. It will be about tools that become part of us; in some cases, part of the illness itself.