Join us for an in-depth conversation with Rahul Rai, an Infrastructure Product Manager at Meta with over a decade of experience spanning Cisco, Samsung, Amazon, Microsoft, and Meta. In this episode, Rahul unpacks the complexities of AI model development—from training to inference—and reveals how product managers can navigate the technical and strategic challenges of building scalable systems that serve billions of users.
What You'll Learn:
• The fundamentals of AI training vs. inference and why distributed inference is a game changer.
• How to balance the needs of multiple stakeholders—from ML engineers to end users—in building robust, scalable products.
• Real-world insights into capacity planning and the creation of internal tools that impact global-scale operations.
• A candid comparison of PM cultures and career paths at Amazon, Microsoft, and Meta, along with tips for transitioning into an infrastructure PM role.
• Recommended resources and actionable advice for any PM looking to excel in high-impact technical roles.
Timestamps:
00:00 – Podcast Intro & Setup
01:46 – Meet Rahul Rai: Background and Career Journey
03:17 – AI 101: Training vs. Inference Fundamentals
06:24 – Deep Dive: Analogies and Real-World Examples of AI Inference
10:03 – Distributed Inference Explained: How Models Stay Current
12:34 – Cost Breakdown: Why Inference Drives 90% of AI Model Costs
17:13 – The Infra PM Role: Balancing Stakeholder Needs
22:43 – Building at Scale: Capacity Planning Tools for Billion-User Platforms
24:36 – Meta’s AI Strategy: Open Source Models and Product Integration
28:28 – Comparing Cultures: PM Roles at Amazon, Microsoft, & Meta
36:50 – Autonomy at Meta: Bottom-Up Problem Solving in Action
40:30 – Growth Opportunities: Essential Skills for Infra PMs
42:46 – Measuring Success: Metrics and Impact in Infrastructure PM
44:32 – Resources for PMs: "First 90 Days" & "Getting Stuff Done"
46:05 – Wrap-Up & Connect: How to Follow Rahul Rai
Recommended Tags:
#ProductManagement #InfrastructurePM #AIInference #DistributedInference #Meta #Amazon #Microsoft #TechLeadership #CapacityPlanning #OpenSourceAI #MachineLearning #ProductManager #TechPodcast
Popular Podcasts
On Purpose with Jay Shetty
I’m Jay Shetty host of On Purpose the worlds #1 Mental Health podcast and I’m so grateful you found us. I started this podcast 5 years ago to invite you into conversations and workshops that are designed to help make you happier, healthier and more healed. I believe that when you (yes you) feel seen, heard and understood you’re able to deal with relationship struggles, work challenges and life’s ups and downs with more ease and grace. I interview experts, celebrities, thought leaders and athletes so that we can grow our mindset, build better habits and uncover a side of them we’ve never seen before. New episodes every Monday and Friday. Your support means the world to me and I don’t take it for granted — click the follow button and leave a review to help us spread the love with On Purpose. I can’t wait for you to listen to your first or 500th episode!
The Breakfast Club
The World's Most Dangerous Morning Show, The Breakfast Club, With DJ Envy And Charlamagne Tha God!
The Joe Rogan Experience
The official podcast of comedian Joe Rogan.