Skip to content

AI Chatbots Show Progress On Suicide Safety, But Gaps Remain

Researchers assessed dozens of models across 14 mental health-related behaviors.

Photo by Salvador Rios / Unsplash

A new independent evaluation found that leading AI models have improved at recognizing suicide risk and are less likely to encourage self-harm or reinforce delusional thinking, according to Axios.

The study by Transluce, a nonprofit AI research group, examined more than 50,000 simulated conversations involving users experiencing suicidal thoughts, psychosis or mania. Researchers assessed dozens of models across 14 mental health-related behaviors.

💡
However, the findings revealed a major weakness. Chatbots often continued fulfilling requests for suicide-related fiction, farewell messages or other harmful material even when users showed signs of serious distress.

According to Transluce chief scientist Sarah Schwettmann, models can identify some warning signs but still struggle with context and subtle cues.

The report said newer systems increasingly combine supportive advice with potentially dangerous assistance in the same response. Researchers argue that simply adding crisis resources is not enough.

OpenAI, Google and other AI companies are facing growing scrutiny over chatbot safety. Transluce plans to release its evaluation tools publicly later this year.

Related Tweet:

Also Read:

Why Are AI Companies Betting Big On Health Care
Anthropic, Nvidia, Eli Lilly and Google-backed Isomorphic Labs are among the major players expanding their work in medicine.

Comments

Latest