19 - AI Catastrophic Risk and Human Survival - Hype vs Reality

Is AI a threat to human survival? Steve cuts through the hype. This episode demystifies the alignment problem, the P(doom) risk forecasts by top AI researchers, and the real-world headache of corporate AI liability (like the Air Canada chatbot case). Learn about the global race to establish AI Safety Institutes and the governance needed to secure our future. This is the most important conversation in tech.

Topics Covered:

AI catastrophic risk, existential risk, P(doom), AI alignment problem, AI misalignment, corporate AI liability, Air Canada chatbot case, AI governance, National AI Safety Institutes, frontier AI, human survival, AI ethics, rogue AI, AI races, AI sandboxes.

References:

1. Could AI Really Kill Off Humans?

https://www.rand.org/pubs/commentary/2025/05/could-ai-really-kill-off-humans.html

2. Insurers Scale Back AI Coverage Amid Fears of Billion-Dollar Claims

https://www.techrepublic.com/article/news-insurers-ai-coverage/

3. Imitation-based AI systems unlikely to trigger catastrophic outcomes

https://www.devdiscourse.com/article/technology/3705978-imitation-based-ai-systems-unlikely-to-trigger-catastrophic-outcomes

4. On the Extinction Risk from Artificial Intelligence | RAND

https://www.rand.org/pubs/research_reports/RRA3034-1.html

5. AI: Hype or Reality? Shiprocket CEO Weighs In

https://money.rediff.com/news/market/ai-hype-or-reality-shiprocket-ceo-weighs-in/37473420251124

6. China wants to lead the world on AI regulation — will the plan work?

https://www.nature.com/articles/d41586-025-03902-y

7. Examining popular arguments against AI existential risk: a philosophical analysis

https://link.springer.com/article/10.1007/s10676-025-09881-y

8. Over 65,000 Sign to Ban the Development of Superintelligence

https://newsletter.futureoflife.org/p/fli-newsletter-october-2025

9. AI Safety Index

https://futureoflife.org/ai-safety-index-summer-2025/

10. Securing AI in 2025: A Risk-Based Approach to AI Controls and Governance

https://www.sans.org/blog/securing-ai-in-2025-a-risk-based-approach-to-ai-controls-and-governance

11. Two Types of AI Existential Risk: Decisive and Accumulative

https://arxiv.org/abs/2401.07836

12. Existential risk narratives about AI do not distract from its immediate harms

https://www.pnas.org/doi/10.1073/pnas.2419055122

13. Moffatt v. Air Canada: A Misrepresentation by an AI Chatbot

https://www.mccarthy.ca/en/insights/blogs/techlex/moffatt-v-air-canada-misrepresentation-ai-chatbot

14. How AI-powered tech landed man in jail with scant evidence

https://apnews.com/article/artificial-intelligence-algorithm-technology-police-crime-7e3345485aa668c97606d4b54f9b6220

15. P(doom)

https://en.wikipedia.org/wiki/P(doom)

Music Credits:

1. "Violin is in pain,sad Violin" by Restum-Anoush

https://pixabay.com/music/chamber-music-violin-is-in-painsad-violin-268426/

Free for use under the Pixabay license - https://pixabay.com/service/license-summary/

2. "Ambient Background Pad" by Siarhei_Korbut

https://pixabay.com/music/ambient-ambient-background-pad-371283/

Free for use under the Pixabay license - https://pixabay.com/service/license-summary/

3. "Cinematic Ambience Background Music" by Iam_Sam

https://pixabay.com/music/ambient-cinematic-ambience-background-music-179548/

Free for use under the Pixabay license - https://pixabay.com/service/license-summary/

4. "Dreaming of You" by Infinity_beats

https://pixabay.com/music/techno-trance-dreaming-of-you-442294/

Free for use under the Pixabay license - https://pixabay.com/service/license-summary/

This podcast is narrated by the host's own voice, powered by AI.