AI Infra for Long Context Model Training | Anna Patterson, founder of Ceramic AI

Anna Patterson is the cofounder of Ceramic, an AI infrastructure platform for large scale model training. They raised their seed round led by NEA along with amazing investors such as Lukas Biewald, Laszlo Bock, Sean Carey, Jeff Hammerbacher, Ankit Jain, Seval Oz, Joanna Rees, Gokul Rajaram, and Ram Sriram. She was previously the founder and managing partner at Gradient Ventures. She was the VP Engineering at Google for 14 years. 

Anna's favorite book: Books she reads with her daughters as part of their family book club

(00:01) Introduction & AI Infra 101
(01:11) Budget Breakdown: Training vs Inference
(02:16) Mapping the AI Infra Landscape
(04:18) Verticalized vs General-Purpose Infrastructure
(06:22) Why Ceramic Was Built From Scratch
(08:35) MVP Tradeoffs and Decision Framework
(10:16) Achieving 2.5x Speedup in Long Context Training
(11:50) Short vs Medium vs Long Context: A Primer
(13:38) Long Context vs RAG (Retrieval-Augmented Generation)
(15:24) Real-World Impact of Long Context Models
(16:38) Bottlenecks at 96K Token Contexts
(17:51) Data Pruning 101: What to Keep, What to Drop
(21:01) What Is “Good Data” in Subjective Domains?
(22:32) How to Grade Reasoning, Not Just Answers
(24:15) Synthetic Data: Use Cases & Limits
(26:19) Staying Current in Fast-Moving Domains
(27:30) Will Every Company Have Its Own Model?
(29:23) Unlocking the Next 10x in Infra
(31:27) Favorite Recent AI Advancements
(32:33) Rapid Fire Round

--------
Where to find Anna Patterson: 

LinkedIn: https://www.linkedin.com/in/anna-patterson-15921ba/

--------
Where to find Prateek Joshi: 

Newsletter: https://prateekjoshi.substack.com 
Website: https://prateekj.com 
LinkedIn: https://www.linkedin.com/in/prateek-joshi-91047b19 
X: https://x.com/prateekvjoshi