054. Jay McClelland: Networks That Learn

TODAY'S GUEST

 

Jay McClelland is a Computational Cognitive Neuroscientist and one of the founding fathers of the field of neural networks and deep learning in the 1980s, which led directly to today's explosion in AI and machine learning algorithms that are transforming our lives. He is the Lucie Stern Professor at Stanford University, where he was formerly the chair of the psychology department, and is currently a Consulting Research Scientist at DeepMind, perhaps the leader in machine learning technologies today.

 

Jay is best known for his work on statistical learning and parallel distributed processing, applying connectionist models (or neural networks) to explain cognitive phenomena such as spoken word recognition and visual word recognition. Today, he works on integrating language, memory, and visuospatial cognition in an integrated understanding system to capture human intelligence and enhance artificial intelligence, exploring how education and human-invented tools of thought can enhance human and machine intelligence. 

 

EPISODE SUMMARY

 

In this conversation we talk about:

• Lessons from his youth, where he moved around the world as a child and interacted with different religions and backgrounds, which helped him understand that we are shaped by our contexts and experiences.
• His entry into cognitive psychology, and going beyond the laws of behavior into: Why do people behave the way they do?
• Building neural networks to model cognition.
• His world-changing PDP paper (Parallel Distributed Processing: Explorations in the Microstructure of Cognition), a paper that was published in 1986 and transformed this whole field, and directly led to more and more people embracing the connectionist model and neural networks.
• The fact and meaning of bi-directionality in neural networks. What does it mean that information can flow both ways in the same network structure?
• Generative models, and in this context, OpenAI's DALL-E 2 algorithm, which can create amazing illustrations and artworks — and should we credit generative or creative algorithms with artistry and give them credit for their art?
• Consciousness — does it extend beyond humans and is it something that we may be able to find someday in algorithms?

 

Talking to Jay really reminded me of the best in mankind, that through curiosity, asking interesting questions, and constructing thought models and experiments, we can unlock such a subtle and fundamental thing like cognition and the connectionist model, which then unlocks all of this power for society at large. We now have this responsibility to reign in the worst of mankind in how we exploit, curate, and share in the benefits of this incredible power. This will be a running topic for us, AI in the future. We explore the power of design and human-centered thinking to create a better future for everyone.

 

This conversation with Jay is one of many weekly conversations we already have lined up for you with leading authors, thinkers, designers, makers, scientists, and social entrepreneurs who are working to change our world for the better. So follow this podcast on your favorite podcast app, or head over to remakepod.org to subscribe.

 

And now, let's jump right in with Jay McClelland.

 

TIMESTAMP CHAPTERS

 

[7:28] Life in the Present

[9:08] Early Childhood Perspectives

[12:33] A Path to Psychology

[22:16] Modeling Cognition

[27:37] Neural Networks

[35:16] The Significance of Bi-Directionality

[40:21] Bistable Perception

[43:55] The Truth of Mathematics

[49:24] An Emergentist

[55:17] Technology and AI

[1:01:17] An Accumulation of Experience

[1:07