OK, we are not experts nor PhDs so most of this is probably not technically correct, but the math is so complicated and the concepts so complicated, that we thought it would be good to just get some intuition on what is happening. So this is a quick talk that summarizes readings from so many different sources about the history of AI from the 1950s all the way to January 2024 or so. It is really hard to keep up but much harder without some intuition about what is happening. We cover expert systems, the emergence of neural networks, Convolution Neural Networks, and Recurrent Neural Networks. The Attention is All You Need paper led to Transformers and then finally some intuition on how such a simple idea, that is training on things can lead to emergent and unexpected behaviors, and finally, some intuition on how Generative images work.
You can go to YouTube to see the slides we are using at YouTube and more information at Tongfamily.com
Chapters:
- 23:25 Attention is all you need Transformers
- 27:36 But it's too Simple! Emergence is surprising
- 33:04 What emerges inside a Transformer?
- 43:01 One Model to Rule Them All
- 47:54 Works for Image generation too
Popular Podcasts
Stuff You Should Know
If you've ever wanted to know about champagne, satanism, the Stonewall Uprising, chaos theory, LSD, El Nino, true crime and Rosa Parks, then look no further. Josh and Chuck have you covered.
Dateline NBC
Current and classic episodes, featuring compelling true-crime mysteries, powerful documentaries and in-depth investigations. Follow now to get the latest episodes of Dateline NBC completely free, or subscribe to Dateline Premium for ad-free listening and exclusive bonus content: DatelinePremium.com
The Male Room with Dr. Jesse Mills
As Director of The Men’s Clinic at UCLA, Dr. Jesse Mills has spent his career helping men understand their bodies, their hormones, and their health. Now he’s bringing that expertise to The Male Room — a podcast where data-driven medicine meets common sense. Each episode separates fact from hype, science from snake oil, and gives men the tools to live longer, stronger, and happier lives. With candor, humor, and real-world experience from the exam room and the operating room, Dr. Mills breaks down the latest health headlines, dissects trends, and explains what actually works — and what doesn’t. Smart, straightforward, and entertaining, The Male Room is the show that helps men take charge of their health without the jargon.