Against 'The Singularity' w/ Dr. David Thorstad

Philosopher Dr. David Thorstad tears into one of AI safety's most influential arguments: the singularity hypothesis. We discuss why the idea of recursive self-improvement leading to superintelligence doesn't hold up under scrutiny, how these arguments have redirected hundreds of millions in funding away from proven interventions, and why people keep backpedaling to weaker versions when challenged.

David walks through the actual structure of singularity arguments, explains why similar patterns show up in other longtermist claims, and makes the case for why we should focus on concrete problems happening right now like poverty, disease, the rise of authoritarianism instead of speculative far-future scenarios.

Chapters

• (00:00) - Intro
• (02:13) - David's background
• (08:00) - (Against) The Singularity Hypothesis
• (29:46) - Beyond the The Singularity
• (39:56) - What We Should Actually Be Worried About
• (49:00) - Philanthropic Funding

Links

• David's personal website
• Reflective Altruism, David's blog

The Singularity Hypothesis

• David's Philosophical Studies article - Against the singularity hypothesis
• Time "AI Dictionary" page - Singularity
• EA Forum blogpost - Summary: Against the singularity hypothesis
• Journal of Conciousness Studies article - The Singularity: A Philisophical Analysis
• Interim Report from the Panel Chairs: AAAI Presidential Panel on Long-Term AI Futures
• Epoch AI blogpost - Do the returns to software R&D point towards a singularity?
• Epoch AI report - Estimating Idea Production: A Methodological Survey

Funding References

• LessWrong blogpost - An Overview of the AI Safety Funding Situation
• AISafety.com funding page
• Report - Stanford AI Index 2025, Chapter 4.3
• Forbes article - AI Spending To Exceed A Quarter Trillion Next Year
• AI Panic article - The “AI Existential Risk” Industrial Complex
• GiveWell webpage - How Much Does It Cost To Save a Life?
• Wikipedia article - Purchasing power parity

Pascal's Mugging and the St. Petersburg Paradox

• Wikipedia article - St. Petersburg Paradox
• Conjecture Magazine article - Pascal’s Mugging and Bad Explanations
• neurabites explainer - Ergodicity: the Most Over-Looked Assumption
• Wikipedia article - Extraordinary claims require extraordinary evidence

The Time of Perils

• Global Priorities Institute working paper - Existential risk pessimism and the time of perils
• Ethics article - Mistakes in the Moral Mathematics of Existential Risk
• Philosophy & Public Affairs article - High Risk, Low Reward: A Challenge to the Astronomical Value of Existential Risk Mitigation
• Toby Ord Book - The Precipice
• Rethink Priorities .css-j9qmi7{display:-webkit-box;display:-webkit-flex;display:-ms-flexbox;display:flex;-webkit-flex-direction:row;-ms-flex-direction:row;flex-direction:row;font-weight:700;margin-bottom:1rem;margin-top:2.8rem;width:100%;-webkit-box-pack:start;-ms-flex-pack:start;-webkit-justify-content:start;justify-content:start;padding-left:5rem;}@media only screen and (max-width: 599px){.css-j9qmi7{padding-left:0;-webkit-box-pack:center;-ms-flex-pack:center;-webkit-justify-content:center;justify-content:center;}}.css-j9qmi7 svg{fill:#27292D;}.css-j9qmi7 .eagfbvw0{-webkit-align-items:center;-webkit-box-align:center;-ms-flex-align:center;align-items:center;color:#27292D;}