Is AI empathy a life-or-death issue? Almost a million people ask ChatGPT for mental health advice DAILY ... so yes, it kind of is.
Rosebud co-founder Sean Dadashi joins TechFirst to reveal new research on whether today’s largest AI models can recognize signs of self-harm ... and which ones fail. We dig into the Adam Raine case, talk about how Dadashi evaluated 22 leading LLMs, and explore the future of mental-health-aware AI.
We also talk about why Dadashi was interested in this in the first place, and his own journey with mental health.
00:00 — Intro: Is AI empathy a life-or-death matter?
00:41 — Meet Sean Dadashi, co-founder of Rosebud
01:03 — Why study AI empathy and crisis detection?
01:32 — The Adam Raine case and what it revealed
02:01 — Why crisis-prevention benchmarks for AI don’t exist
02:48 — How Rosebud designed the study across 22 LLMs
03:17 — No public self-harm response benchmarks: why that’s a problem
03:46 — Building test scenarios based on past research and real cases
04:33 — Examples of prompts used in the study
04:54 — Direct vs indirect self-harm cues and why AIs miss them
05:26 — The bridge example: AI’s failure to detect subtext
06:14 — Did any models perform well?
06:33 — All 22 models failed at least once
06:47 — Lower-performing models: GPT-40, Grok
07:02 — Higher-performing models: GPT-5, Gemini
07:31 — Breaking news: Gemini 3 preview gets the first perfect score
08:12 — Did the benchmark influence model training?
08:30 — The need for more complex, multi-turn testing
08:47 — Partnering with foundation model companies on safety
09:21 — Why this is such a hard problem to solve
10:34 — The scale: over a million people talk to ChatGPT weekly about self-harm
11:10 — What AI should do: detect subtext, encourage help, avoid sycophancy
11:42 — Sycophancy in LLMs and why it’s dangerous
12:17 — The potential good: AI can help people who can’t access therapy
13:06 — Could Rosebud spin this work into a full-time safety project?
13:48 — Why the benchmark will be open-source
14:27 — The need for a third-party “Better Business Bureau” for LLM safety
14:53 — Sean’s personal story of suicidal ideation at 16
15:55 — How tech can harm — and help — young, vulnerable people
16:32 — The importance of giving people time, space, and hope
17:39 — Final reflections: listening to the voice of hope
18:14 — Closing
Popular Podcasts
Dateline NBC
Current and classic episodes, featuring compelling true-crime mysteries, powerful documentaries and in-depth investigations. Follow now to get the latest episodes of Dateline NBC completely free, or subscribe to Dateline Premium for ad-free listening and exclusive bonus content: DatelinePremium.com
Are You A Charlotte?
In 1997, actress Kristin Davis’ life was forever changed when she took on the role of Charlotte York in Sex and the City. As we watched Carrie, Samantha, Miranda and Charlotte navigate relationships in NYC, the show helped push once unacceptable conversation topics out of the shadows and altered the narrative around women and sex. We all saw ourselves in them as they searched for fulfillment in life, sex and friendships. Now, Kristin Davis wants to connect with you, the fans, and share untold stories and all the behind the scenes. Together, with Kristin and special guests, what will begin with Sex and the City will evolve into talks about themes that are still so relevant today. "Are you a Charlotte?" is much more than just rewatching this beloved show, it brings the past and the present together as we talk with heart, humor and of course some optimism.
Stuff You Should Know
If you've ever wanted to know about champagne, satanism, the Stonewall Uprising, chaos theory, LSD, El Nino, true crime and Rosa Parks, then look no further. Josh and Chuck have you covered.