In this illuminating episode of The Cognitive Revolution, host Nathan Labenz speaks with Jack Rae, principal research scientist at Google DeepMind and technical lead on Google's thinking and inference time scaling work. They explore the technical breakthroughs behind Google's Gemini 2.5 Pro model, discussing why reasoning techniques are suddenly working so effectively across the industry and whether these advances represent true breakthroughs or incremental progress. The conversation delves into critical questions about the relationship between reasoning and agency, the role of human data in shaping model behavior, and the roadmap from current capabilities to AGI, providing listeners with an insider's perspective on the trajectory of AI development.
SPONSORS:
Oracle Cloud Infrastructure (OCI): Oracle Cloud Infrastructure offers next-generation cloud solutions that cut costs and boost performance. With OCI, you can run AI projects and applications faster and more securely for less. New U.S. customers can save 50% on compute, 70% on storage, and 80% on networking by switching to OCI before May 31, 2024. See if you qualify at https://oracle.com/cognitive
Shopify: Shopify is revolutionizing online selling with its market-leading checkout system and robust API ecosystem. Its exclusive library of cutting-edge AI apps empowers e-commerce businesses to thrive in a competitive market. Cognitive Revolution listeners can try Shopify for just $1 per month at https://shopify.com/cognitive
NetSuite: Over 41,000 businesses trust NetSuite by Oracle, the #1 cloud ERP, to future-proof their operations. With a unified platform for accounting, financial management, inventory, and HR, NetSuite provides real-time insights and forecasting to help you make quick, informed decisions. Whether you're earning millions or hundreds of millions, NetSuite empowers you to tackle challenges and seize opportunities. Download the free CFO's guide to AI and machine learning at https://netsuite.com/cognitive
PRODUCED BY:
https://aipodcast.ing
CHAPTERS:
(00:00) About the Episode
(05:09) Introduction and Welcome
(07:28) RL for Reasoning
(10:46) Research Time Management
(13:41) Convergence in Model Development
(18:31) RL on Smaller Models (Part 1)
(20:01) Sponsors: Oracle Cloud Infrastructure (OCI) | Shopify
(22:35) RL on Smaller Models (Part 2)
(23:30) Sculpting Cognitive Behaviors
(25:05) Language Switching Behavior
(28:02) Sharing Chain of Thought
(32:03) RL on Chain of Thought (Part 1)
(33:46) Sponsors: NetSuite
(35:19) RL on Chain of Thought (Part 2)
(35:26) Eliciting Human Reasoning
(39:27) Reasoning vs. Agency
(40:17) Understanding Model Reasoning
(44:29) Reasoning in Latent Space
(47:54) Interpretability Challenges
(51:36) Platonic Model Hypothesis
(56:05) Roadmap to AGI
(01:00:57) Multimodal Integration
(01:04:38) System Card Questions
(01:07:51) Long Context Capabilities
(01:13:49) Outro