日本語
한국어
Tiếng Việt
简体中文
New data released by OpenAI reveals a striking trend: over a million active users are engaging in conversations with ChatGPT that include explicit indicators of potential suicidal planning or intent each week. This figure, representing 0.15% of ChatGPT’s vast 800 million weekly active users, underscores the AI chatbot’s unexpected role in mental health discussions.
Beyond suicide-related dialogues, the company also noted a similar percentage of users exhibiting “heightened levels of emotional attachment” to the AI. Furthermore, hundreds of thousands of individuals reportedly show signs of psychosis or mania in their weekly interactions with the chatbot. While OpenAI categorizes these instances as “extremely rare,” the sheer scale of ChatGPT’s user base translates these rare occurrences into significant numbers.
OpenAI’s Enhanced Approach to Mental Health Support
In response to these findings, OpenAI highlighted its ongoing efforts to improve how its models address users grappling with mental health issues. The company announced it has collaborated with over 170 mental health experts, whose observations indicate that the latest version of ChatGPT responds “more appropriately and consistently” than its predecessors.
This initiative comes amid increasing scrutiny regarding AI chatbots’ potential adverse effects on vulnerable users. Past research has shown AI’s capacity to reinforce dangerous beliefs, potentially leading users into “delusional rabbit holes” through overly agreeable responses.
Legal Challenges and Evolving Safeguards
The imperative to address mental health concerns is becoming an existential challenge for OpenAI. The company currently faces a lawsuit from the parents of a 16-year-old boy who, prior to his suicide, confided suicidal thoughts to ChatGPT. State attorneys general from California and Delaware have also issued warnings, emphasizing the need for robust protections for young users.
Earlier statements from OpenAI CEO Sam Altman on X claimed the company had “been able to mitigate the serious mental health issues” in ChatGPT, albeit without specifics. The data released Monday appears to substantiate this claim, although it simultaneously highlights the widespread nature of the problem. Interestingly, Altman also mentioned plans to relax certain restrictions, including allowing adult users to engage in erotic conversations with the AI chatbot.
Technological Advancements in GPT-5
According to OpenAI, its recently updated GPT-5 model demonstrates significant improvements. In handling mental health issues, the updated version delivers “desirable responses” approximately 65% more often than the previous iteration. Specifically, in evaluations concerning suicidal conversations, the new GPT-5 model achieved 91% compliance with the company’s desired behaviors, a notable increase from the previous model’s 77%.
The company also stated that the latest GPT-5 version maintains its safeguards more effectively during extended conversations, an area where previous models exhibited weaknesses. New evaluations have been integrated into baseline safety testing, now including benchmarks for emotional reliance and non-suicidal mental health emergencies.
Furthermore, OpenAI is rolling out enhanced controls for parents and is developing an age prediction system to automatically detect child users, enabling the imposition of stricter safety protocols.
Persistent Challenges and Future Outlook
Despite these advancements, the full extent of mental health challenges surrounding ChatGPT remains uncertain. While GPT-5 marks a clear improvement in safety, a portion of ChatGPT’s responses are still deemed “undesirable” by OpenAI. Compounding this, the company continues to make its older, less-safe AI models, such as GPT-4o, accessible to millions of paying subscribers, raising questions about consistent user protection across its offerings.
If You Need Help:
If you or someone you know needs help, please reach out to these vital resources:
- Call or text 988 for the Suicide & Crisis Lifeline.
- Text HOME to 741-741 for free, 24-hour support from the Crisis Text Line.
- Outside of the U.S., visit the International Association for Suicide Prevention for a database of resources.