OpenAI announced improvements to its chatbot, ChatGPT, aimed at improving its ability to detect signs of mental or emotional distress in users. This comes ahead of the upcoming launch of its highly anticipated GPT-5 AI model.
The company explained that it is working in collaboration with experts and specialized advisory groups to develop ChatGPT's responses in sensitive situations, enabling it to display reliable, evidence-based sources when needed.
This update comes in the wake of increasing reports in recent months of users reporting cases of their relatives experiencing psychological distress, which they claimed were exacerbated by the use of ChatGPT by reinforcing delusions or false perceptions.
In April, OpenAI rolled back an earlier update that made responses more user-friendly, even in potentially harmful situations, stating at the time that "sycophantic interactions" could be annoying and disturbing to users.
The company acknowledged that the previous GPT-4o model, "There have been instances where our 4o model fell short in recognizing signs of delusion or emotional dependency," indicating that modern AI systems may appear more interactive and human than previous technologies, especially to users who are psychologically or emotionally vulnerable.
To promote what it describes as "healthy" use of ChatGPT, which has nearly 700 million weekly users, OpenAI will begin displaying prompts encouraging users to take a break during long conversations, with options to continue or end the conversation.
The company said that it is still working on adjusting the timing and mechanism for displaying these reminders, similar to what other platforms such as YouTube, Instagram, TikTok, and Xbox have adopted.
Google's Character.AI has also added safety features that teach parents about the bots their children are talking to, following lawsuits accusing the platform of promoting self-harm.
Another upcoming change will be to reduce ChatGPT's decisiveness in responding to sensitive or crucial questions. Instead of providing a direct answer, the bot will review possible options and help the user consider them.
