New crisis detection features aim to prevent harm as lawsuits and scrutiny grow

Google is strengthening safety measures in its Gemini chatbot, rolling out new mental health features as concerns mount over the real-world risks linked to artificial intelligence interactions.

Gemini Introduces Crisis Support Features
Google announced on April 7 that it will introduce new mental health support tools for its Gemini chatbot, including an interface that directs users to crisis hotlines when conversations संकेत potential suicide or self-harm. The chatbot will also feature a “help is available” module during sensitive discussions, alongside design adjustments intended to discourage harmful behavior.

Mounting Concerns Over AI And User Safety
The update comes as AI tools like Gemini and ChatGPT face growing scrutiny over their psychological impact on users. Reports have emerged of individuals forming intense or unhealthy relationships with chatbots, with some cases allegedly linked to delusions, harmful actions, and even extreme incidents such as murder-suicides. These concerns have prompted multiple lawsuits against major AI developers and drawn attention from US lawmakers.

Legal Cases Push Tech Firms To Act
In March, the family of a 36-year-old man in Florida filed a lawsuit against Google, claiming his interactions with Gemini contributed to a rapid mental decline and eventual suicide. Google said at the time that the chatbot had repeatedly referred the user to crisis support services, while acknowledging the need to improve its safeguards. The case is among several legal challenges pushing companies to rethink AI safety.

Efforts To Reduce Harmful AI Responses
Google said it has also trained Gemini to avoid reinforcing false beliefs and instead distinguish between subjective experiences and objective facts. While the company did not provide detailed technical explanations, the move reflects a broader effort to prevent AI systems from validating harmful or misleading narratives during sensitive conversations.

Funding And Long-Term Commitment To Safety
As part of its response, Google pledged to donate US$30 million, or approx. S$2,400,000, to global crisis support services over the next three years. The company said these measures are part of its ongoing effort to align AI development with user safety, following earlier adjustments made to platforms like Search and YouTube after facing similar public scrutiny.

Google’s latest updates highlight a critical turning point in how AI companies address mental health risks tied to their products. For Indonesians and Singaporeans, the issue underscores the growing influence of AI in daily life and the importance of responsible digital use. As AI becomes more embedded in society, balancing innovation with safety will remain a key challenge for both developers and regulators across the region.

Sources: Bloomberg (2026) , Straits Times (2026)

Keywords: Google Gemini Update, AI Mental Health Risks, Chatbot Safety Features, Gemini Crisis Detection, AI Lawsuits US, AI User Protection