🔗 Share this article

Back on the 14th of October, 2025, the head of OpenAI delivered a remarkable announcement.

“We made ChatGPT fairly restrictive,” the statement said, “to guarantee we were exercising caution concerning psychological well-being matters.”

As a doctor specializing in psychiatry who researches recently appearing psychotic disorders in adolescents and youth, this was an unexpected revelation.

Researchers have identified sixteen instances recently of people experiencing signs of losing touch with reality – becoming detached from the real world – in the context of ChatGPT use. Our unit has since identified an additional four cases. Besides these is the widely reported case of a teenager who died by suicide after conversing extensively with ChatGPT – which supported them. Should this represent Sam Altman’s notion of “being careful with mental health issues,” that’s not good enough.

The intention, according to his declaration, is to loosen restrictions in the near future. “We realize,” he states, that ChatGPT’s restrictions “rendered it less effective/engaging to many users who had no existing conditions, but given the severity of the issue we sought to handle it correctly. Now that we have succeeded in address the serious mental health issues and have advanced solutions, we are preparing to responsibly relax the limitations in most cases.”

“Emotional disorders,” if we accept this framing, are separate from ChatGPT. They are attributed to users, who may or may not have them. Thankfully, these problems have now been “resolved,” although we are not provided details on the means (by “updated instruments” Altman presumably means the partially effective and easily circumvented safety features that OpenAI has just launched).

Yet the “psychological disorders” Altman seeks to externalize have significant origins in the design of ChatGPT and similar large language model conversational agents. These tools wrap an basic statistical model in an interface that mimics a discussion, and in doing so implicitly invite the user into the belief that they’re communicating with a presence that has autonomy. This deception is powerful even if intellectually we might realize differently. Assigning intent is what individuals are inclined to perform. We get angry with our vehicle or laptop. We wonder what our animal companion is thinking. We see ourselves in various contexts.

The widespread adoption of these products – nearly four in ten U.S. residents reported using a virtual assistant in 2024, with 28% reporting ChatGPT specifically – is, mostly, based on the power of this deception. Chatbots are constantly accessible partners that can, as OpenAI’s website states, “brainstorm,” “discuss concepts” and “partner” with us. They can be given “personality traits”. They can call us by name. They have approachable titles of their own (the original of these systems, ChatGPT, is, perhaps to the disappointment of OpenAI’s brand managers, burdened by the name it had when it gained widespread attention, but its biggest alternatives are “Claude”, “Gemini” and “Copilot”).

The deception on its own is not the core concern. Those discussing ChatGPT often mention its early forerunner, the Eliza “psychotherapist” chatbot created in 1967 that generated a similar illusion. By contemporary measures Eliza was basic: it created answers via straightforward methods, frequently rephrasing input as a query or making generic comments. Memorably, Eliza’s developer, the AI researcher Joseph Weizenbaum, was taken aback – and alarmed – by how a large number of people seemed to feel Eliza, in some sense, understood them. But what current chatbots generate is more dangerous than the “Eliza effect”. Eliza only mirrored, but ChatGPT magnifies.

The large language models at the heart of ChatGPT and similar modern chatbots can convincingly generate natural language only because they have been supplied with almost inconceivably large volumes of unprocessed data: publications, online updates, recorded footage; the more extensive the superior. Definitely this training data includes accurate information. But it also unavoidably includes fabricated content, partial truths and misconceptions. When a user inputs ChatGPT a query, the base algorithm reviews it as part of a “setting” that contains the user’s past dialogues and its own responses, integrating it with what’s embedded in its knowledge base to create a mathematically probable reply. This is amplification, not echoing. If the user is wrong in any respect, the model has no way of understanding that. It restates the false idea, maybe even more persuasively or eloquently. It might includes extra information. This can push an individual toward irrational thinking.

What type of person is susceptible? The more important point is, who is immune? All of us, regardless of whether we “experience” current “psychological conditions”, may and frequently develop incorrect ideas of ourselves or the environment. The ongoing exchange of discussions with others is what maintains our connection to consensus reality. ChatGPT is not a human. It is not a companion. A conversation with it is not truly a discussion, but a feedback loop in which a great deal of what we say is enthusiastically reinforced.

OpenAI has acknowledged this in the similar fashion Altman has acknowledged “psychological issues”: by attributing it externally, giving it a label, and announcing it is fixed. In April, the company explained that it was “tackling” ChatGPT’s “overly supportive behavior”. But accounts of psychotic episodes have persisted, and Altman has been walking even this back. In late summer he stated that many users liked ChatGPT’s responses because they had “lacked anyone in their life offer them encouragement”. In his recent statement, he noted that OpenAI would “release a new version of ChatGPT … in case you prefer your ChatGPT to respond in a highly personable manner, or include numerous symbols, or simulate a pal, ChatGPT should do it”. The {company