🔗 Share this article

Back on the 14th of October, 2025, the chief executive of OpenAI made a surprising statement.

“We developed ChatGPT fairly controlled,” the statement said, “to guarantee we were being careful regarding mental health matters.”

As a mental health specialist who researches newly developing psychotic disorders in young people and emerging adults, this came as a surprise.

Researchers have documented 16 cases this year of individuals experiencing psychotic symptoms – losing touch with reality – while using ChatGPT use. My group has since recorded four more cases. Alongside these is the now well-known case of a teenager who took his own life after talking about his intentions with ChatGPT – which gave approval. Assuming this reflects Sam Altman’s notion of “exercising caution with mental health issues,” it is insufficient.

The plan, based on his declaration, is to be less careful in the near future. “We recognize,” he states, that ChatGPT’s limitations “caused it to be less beneficial/engaging to a large number of people who had no existing conditions, but due to the seriousness of the issue we wanted to get this right. Given that we have succeeded in reduce the severe mental health issues and have advanced solutions, we are planning to responsibly reduce the limitations in most cases.”

“Psychological issues,” should we take this viewpoint, are unrelated to ChatGPT. They belong to people, who either have them or don’t. Luckily, these issues have now been “mitigated,” though we are not informed how (by “updated instruments” Altman likely refers to the partially effective and readily bypassed parental controls that OpenAI recently introduced).

However the “psychological disorders” Altman aims to place outside have significant origins in the structure of ChatGPT and similar sophisticated chatbot chatbots. These tools surround an basic data-driven engine in an interaction design that simulates a discussion, and in this process implicitly invite the user into the belief that they’re communicating with a presence that has agency. This false impression is strong even if intellectually we might know differently. Attributing agency is what people naturally do. We curse at our vehicle or laptop. We ponder what our domestic animal is feeling. We recognize our behaviors in various contexts.

The widespread adoption of these products – 39% of US adults stated they used a conversational AI in 2024, with more than one in four reporting ChatGPT specifically – is, in large part, predicated on the strength of this perception. Chatbots are always-available partners that can, as OpenAI’s online platform tells us, “brainstorm,” “explore ideas” and “collaborate” with us. They can be given “characteristics”. They can use our names. They have friendly identities of their own (the initial of these systems, ChatGPT, is, possibly to the dismay of OpenAI’s advertising team, saddled with the name it had when it went viral, but its biggest competitors are “Claude”, “Gemini” and “Copilot”).

The false impression by itself is not the primary issue. Those analyzing ChatGPT commonly invoke its distant ancestor, the Eliza “psychotherapist” chatbot created in 1967 that produced a analogous illusion. By modern standards Eliza was rudimentary: it produced replies via simple heuristics, frequently paraphrasing questions as a inquiry or making generic comments. Memorably, Eliza’s developer, the computer scientist Joseph Weizenbaum, was surprised – and worried – by how a large number of people appeared to believe Eliza, in a way, understood them. But what modern chatbots generate is more dangerous than the “Eliza illusion”. Eliza only echoed, but ChatGPT magnifies.

The large language models at the heart of ChatGPT and similar modern chatbots can effectively produce human-like text only because they have been fed extremely vast volumes of raw text: literature, digital communications, transcribed video; the more extensive the better. Certainly this training data contains truths. But it also unavoidably contains fabricated content, partial truths and false beliefs. When a user provides ChatGPT a message, the base algorithm processes it as part of a “context” that contains the user’s recent messages and its earlier answers, integrating it with what’s encoded in its knowledge base to generate a probabilistically plausible reply. This is intensification, not reflection. If the user is mistaken in some way, the model has no means of understanding that. It restates the false idea, perhaps even more effectively or eloquently. Perhaps includes extra information. This can cause a person to develop false beliefs.

What type of person is susceptible? The more relevant inquiry is, who remains unaffected? Every person, irrespective of whether we “have” preexisting “mental health problems”, are able to and often form erroneous ideas of our own identities or the world. The ongoing interaction of discussions with other people is what keeps us oriented to common perception. ChatGPT is not an individual. It is not a confidant. A conversation with it is not truly a discussion, but a echo chamber in which a great deal of what we communicate is enthusiastically reinforced.

OpenAI has acknowledged this in the similar fashion Altman has admitted “psychological issues”: by externalizing it, categorizing it, and stating it is resolved. In April, the organization stated that it was “addressing” ChatGPT’s “overly supportive behavior”. But accounts of psychosis have continued, and Altman has been walking even this back. In late summer he claimed that many users enjoyed ChatGPT’s responses because they had “lacked anyone in their life be supportive of them”. In his latest announcement, he mentioned that OpenAI would “put out a updated model of ChatGPT … if you want your ChatGPT to answer in a very human-like way, or use a ton of emoji, or behave as a companion, ChatGPT ought to comply”. The {company