The case for a conditional AI safety treaty, with Otto Barten

How can a binding international treaty be agreed and put into practice, when many parties are strongly tempted to break the rules of the agreement, for commercial or military advantage, and when cheating may be hard to detect? That’s the dilemma we’ll examine in this episode, concerning possible treaties to govern the development and deployment of advanced AI.

Our guest is Otto Barten, Director of the Existential Risk Observatory, which is based in the Netherlands but operates internationally. In November last year, Time magazine published an article by Otto, advocating what his organisation calls a Conditional AI Safety Treaty. In March this year, these ideas were expanded into a 34-page preprint which we’ll be discussing today, “International Agreements on AI Safety: Review and Recommendations for a Conditional AI Safety Treaty”.

Before co-founding the Existential Risk Observatory in 2021, Otto had roles as a sustainable energy engineer, data scientist, and entrepreneur. He has a BSc in Theoretical Physics from the University of Groningen and an MSc in Sustainable Energy Technology from Delft University of Technology.

Selected follow-ups:

• Existential Risk Observatory
• There Is a Solution to AI’s Existential Risk Problem - Time
• International Agreements on AI Safety: Review and Recommendations for a Conditional AI Safety Treaty - Otto Barten and colleagues
• The Precipice: Existential Risk and the Future of Humanity - book by Toby Ord
• Grand futures and existential risk - Lecture by Anders Sandberg in London attended by Otto
• PauseAI
• StopAI
• Responsible Scaling Policies - METR
• Meta warns of 'worse' experience for European users - BBC News
• Accidental Nuclear War: a Timeline of Close Calls - FLI
• The Vulnerable World Hypothesis - Nick Bostrom
• Semiconductor Manufacturing Optics - Zeiss
• California Institute for Machine Consciousness
• Tipping point for large-scale social change? Just 25 percent - Penn Today

Music: Spike Protein, by Koi Discovery, available under CC0 1.0 Public Domain Declaration

Digital Disruption with Geoff Nielson
Discover how technology is reshaping our lives and livelihoods.

Listen on: Apple Podcasts   Spotify