February 2, 2026 • 58 mins
Is intelligence a property of individual brains, or is it something that emerges from many brains trying to align with one another? How can we build AI agents to improve our understanding of the world and to mediate between rivaling humans? For this and much more, we speak today with Danielle Perszyk, a cognitive scientist who leads the human-computer interaction team at Amazon’s AGI Lab.
Listen
Watch
Mark as Played
Transcript
Episode Transcript
Available transcripts are automatically generated. Complete accuracy is not guaranteed.
Speaker 1 (00:05):
Is it possible that we're thinking about intelligence in the
wrong way? Instead of being something inside individual brains, is
intelligence instead something that emerges from lots of brains that
are constantly working to align with one another. And if
we take on that lens, what does this mean about
(00:26):
the way that we can build AI agents or the
way that they can make us better? What is the
difference between information and information with a purpose? Today we're
going to speak with Daniel Persick, a cognitive scientist who
leads the Human Computer Interaction team at Amazon's AGI Lab.
(00:47):
So get ready for a great brain stretch. Welcome to
Intercosmos with me David Eagleman. I'm a neuroscientist and an
author at Stanford and in these episodes, as we sail
deeply into our three pound universe to understand how we
see the world and soon how AI might come to
(01:08):
understand the world with us. Let's think about the word intelligence.
You might justifiably assume that neuroscientists have an agreed upon
(01:30):
definition for this, but we actually don't. However one thinks
about intelligence, I think it's a fair assumption that most
of us, when we think about it, assume that intelligence
is something that happens inside a single head, in other words,
a brain processing information. This statement seems so obvious that
(01:52):
it hardly invites inspection, but if you step back and
look at how intelligence actually unfolds in a human life,
a different picture can start to emerge. Our thinking is
shaped by other people from the very beginning. We learn
by watching, by imitating, by trying to communicate, and eventually
(02:15):
by negotiating meaning with the people around us. Even our
most private thoughts are built from tools that are fundamentally social,
things like language and symbols and shared concepts and cultural norms.
So this may sound strange, but this is what we're
going to talk about today, and the idea will become
very clear. Most of humanity's greatest achievements didn't come from
(02:39):
lone geniuses working in isolation, but from really dense networks
of minds interacting over time. When we look at things
like science or art, or morality or technology, it almost
never makes sense to interpret these as products of individual intelligence,
but instead they are collective processes that allow ideas to
(03:04):
collide and to form into something and to continuously evolve.
So intelligence in this sense may be less like a
thing we possess and more like something that emerges between us. Now,
this broader perspective becomes especially important as we find ourselves
(03:26):
flinging headlong into the era of artificial intelligence. With every
passing week, we're getting AI acting more like a participant.
We're getting systems that communicate but also agents that act
on our behalf to do things in the world. And
soon these agents will collaborate with each other at their
(03:49):
time scales and spatial scales. So if intelligence is social
by nature, then building the future world of AI might
end up requiring more than just dumping billions into scaling
up the training data for these systems. It may instead
require understanding how minds relate to one another in the
(04:13):
first place. And that's where today's conversation begins. Today I'm
joined by Danielle Persk. She's a cognitive scientist who leads
the Human Computer Interaction team at Amazon's AGI Lab. Danielle
uses insights from the evolution and development of human intelligence
to inform how we can not only make AI smarter,
(04:36):
but build AI that also makes us smarter. Here's my
conversation with Danielle Persk.
Speaker 2 (04:47):
Intelligence in humans is really social, and that is the
thing that differentiates our intelligence from other species. Even other
species that are closely related to us have similar brain
structures and function similar genetics. And what we are really
optimizing for is representing other minds. So not only are
(05:10):
infants human infants inferring the existence of other minds, but
once this thing exists, we are optimized for aligning our minds. Evolutionarily,
we had to cooperate to survive. Infants need to be
able to have their caretaker's attention on them to survive,
and in terms of being able to learn about the world,
(05:33):
once infants have a model of other minds, then they
can manipulate it. They can direct their caretaker's attention point
what's that, and magically they'll have a label for.
Speaker 3 (05:41):
This thing that they're looking at in their environment.
Speaker 1 (05:43):
So they're doing prompt engineering.
Speaker 3 (05:47):
Great technology. Yeah, okay, so we know that.
Speaker 2 (05:50):
You know, throughout the course of human evolution, we became
increasingly dependent upon cooperating to to stay alive and adapt
to new environment. So it makes sense that there'd be
this extreme pressure on being able to predict each other's
behaviors to understand our minds, and then with infants, developmentally,
we have also the benefit of being able to learn
(06:13):
much more efficiently even language itself, from representing other minds.
Speaker 1 (06:19):
Okay, so it turns out that we can do a
much better job of predicting if we can imagine what
it's like to be inside other people's heads. Right, So,
if I want to know what some non player character
is going to do in a video game whatever, they
have certain behaviors. But if I want to know, let's
say what you're going to do next, or say next,
(06:39):
if I have a model of your mind and what
you know and you don't know and all that stuff,
I can make a better prediction.
Speaker 2 (06:45):
And so you've said that there's information and then information
with a purpose, and that information with a purpose really matters.
Speaker 3 (06:53):
So you've used the example of like the.
Speaker 2 (06:56):
Land rover on Mars not being able to fix itself,
and like a wolf that gets its like trap.
Speaker 1 (07:03):
Right, the Curiosity Rover went up to Mars. We had
spent like a billion something dollars on it. It did
a great job on Mars, but eventually it got its
right front wheel stuck in the Martian soil and it
died couldn't get out. But if you can trast that
with a wolf who gets its leg cond of trap.
It'll chew its leg off and then figure out how
to walk on three legs, which is extraordinary because a
(07:24):
wolf's brain didn't evolve for three legs. But it can
figure it out because it's live wired. It has brain
plasticity and figure out, Okay, how do I adjust everything.
Speaker 3 (07:33):
So that I can survival?
Speaker 1 (07:34):
Depends upon it exactly. That's the key. It has relevance
to the animal.
Speaker 2 (07:39):
Right, So all animals have a drive to survive, a
drive to reproduce, But humans also have a drive to
align our minds because it helps us cooperate, it helps
us survive, and it helps us to learn extremely efficiently.
So we don't just model other minds. That would just
(07:59):
be the information part. We are optimized for aligning our minds.
So it's information with a purpose.
Speaker 1 (08:05):
Okay, so aligning our minds this is the key thing
and at the center of your interests. And so then
you went into looking into AGI. So first of all,
tell us what artificial general intelligence is to you.
Speaker 2 (08:18):
Well, I think most of the labs that are trying
to build something like AGI, they all have their own definitions.
None of them are really very good. But the one
thing that unifies all of them is that they are
all benchmarked to human intelligence. And this goes all the
way back to the origin of the field of AI.
(08:39):
So in nineteen fifty six, a group of these engineers
and mathematicians got together. They were going to solve intelligence
and build thinking machines, and the idea is that these
thinking machines would think like us.
Speaker 3 (08:49):
It obviously took a.
Speaker 2 (08:50):
Very long time to realize, Oh, that's a lot harder
than we thought that it was. But now we are
back to aiming for something like that original goal of
building thinking machines that think like us.
Speaker 3 (09:01):
We call it AGI.
Speaker 2 (09:03):
Again, have slightly different operationalizations. But I think that we're
all running towards the wrong thing. And that's because I
don't think that intelligence can exist in a machine. It
doesn't exist in individual humans. It's something that emerges from
our interactions because we have this drive to align our representations,
(09:27):
and of course we all have very different representations.
Speaker 3 (09:30):
Right When I used to teach.
Speaker 2 (09:31):
Cognitive science, I would teach about this condition called a fantasia,
and once every couple of classes a student would come
up to.
Speaker 1 (09:40):
Me quick In fantations where you can't imagine, you don't
have any visual representation on the Yes.
Speaker 2 (09:45):
Yes, a student would come up to me and say, wait,
you mean there are people who can actually imagine things.
Speaker 3 (09:51):
Their mind's eye is not just a metaphor. It's a
thing that.
Speaker 2 (09:53):
People experience, and they wouldn't know because they don't suffer
from other types of death. It's just one of the
many ways in which human cognition and experience can very
And when I imagine in apple, it's different than when
you imagine an apple. We all have different associations. So
(10:15):
when we come together and we have to use words
to try to align our minds, there's necessarily going to
be friction, especially when we're trying to talk about abstract things,
especially when we're talking about things at the bleeding edge
of our knowledge, like science.
Speaker 3 (10:34):
How do you align.
Speaker 2 (10:37):
Your representations when there's not even a word for something.
So intelligence emerges as a function of trying to align
our minds and oftentimes creating new concepts to achieve that.
Speaker 1 (10:51):
Okay, so when you're talking about aligning minds, it's because
I've got my whole internal world. You've got your whole
internal world that is built by each of our our
trajectories through space time. We've had different experiences all these things.
So we come together and we've got completely different worlds
running on the inside. And that's what conversation is about.
We're trying to align things that way.
Speaker 2 (11:12):
And there are neuroscientists who measure when people are either
communicating in real time or if they're listening to a story,
if they're watching something on a screen, you can measure
the degree of neuralsynchrony, how close they are to be
on the same wavelength, and that predicts all sorts of
things like how much they like each other, how much
(11:32):
they understood the story, and how much they liked the story,
how similarly they remember things.
Speaker 1 (11:39):
Okay, so this is what humans do. We get together
in conversation all the time and we try to achieve
that synchrony in terms of oh, okay, wait, you have
a different view than I do on this, here's how
we can make progress. This is the Socratic dialectic, right.
This is what Socrates love to do, is have these
conversations where the truth emerges, something bigger than either person
(12:00):
knew when they started the conversation.
Speaker 2 (12:03):
And on that point too, I think a lot of
us think that we know things, but actually when we're
forced to describe something we realized we don't.
Speaker 1 (12:14):
Yeah. Actually, in my next book, I'm talking about this
as a Potempkin village. Yeah, so you know. The Potemkin village,
for anyone doesn't remember, is when it was Catherine the
Great of Russia was heading down the river with a
bunch of dignitaries that she was trying to impress. She
hired this skuy Potempkin. Actually he was her lover as
well as a military general, but she got him to
(12:34):
go down the river a long way and build what
looked like a facade of a village so that when
the ship went by, all the dignitaries would be impressed
that there was this village. And he got all these
peasants like walk around happily and stuff, But there were
no buildings. It was just the front face of the building.
And then when the ship passed, he deconstructed this and
(12:57):
went ahead and built another village so that they passed
another great village, so it looked like things were really
happening there. Anyway, Cognition is often like this, where we think, oh, yeah,
I got it. Here's an example that I often use
is for anybody listening, take out a piece of paper,
and draw a bicycle, draw a bike.
Speaker 3 (13:16):
I've tried this, yeah hard, Yeah.
Speaker 1 (13:19):
Exactly, it turns out, I mean something as simple as
a bike what you see every day. Yeah, you start realizing, wait,
actually I don't know exactly where this goes and what's
the thing and so anyway, Yes, this is an example
of where we think we have deep knowledge and sometimes
it's just the facade of something that we know.
Speaker 2 (13:35):
Yeah, and you can apply it on all different levels.
So you're describing, like the visual imagery might not be
very stable, but a lot of concepts are not stable either,
and we invent ways of making them more stable. Words
are a classic example of that. Once you have a
word for something, you can more easily trigger it, you
(13:57):
can more easily remember it, you can use it and
manipulate it and apply it to different things. But there's
a whole class of things that we're constantly inventing to
better align our minds. They're called cognitive technologies. So writing
would be one of the original ones. But symbols like math, logic, Yeah.
Speaker 1 (14:17):
So unpack that. What's an example of this?
Speaker 3 (14:19):
So literally, any word that you learn. Let's go back
to apples.
Speaker 2 (14:23):
So children see apples, they don't have a word associated
with it, so the likelihood that it's going to spontaneously
sort of emerge in their mind is very low. Maybe
if they've seen a couple then there will be some
sort of increased likelihood or lowered threshold. But once they
(14:44):
have a word for that thing, that they reliably associate
it with it, then anybody who says the word anytime
they hear it, now their brain will elicit that activity
and it becomes a more stable representation. That's an obvious
example where where there's actually a physical thing that the
word can refer to.
Speaker 3 (15:03):
But what about.
Speaker 2 (15:04):
Concepts like love and justice that you can't see you're
saying By assigning a word to it, then we make
that stable, and then it become it can become associated
with a whole web of other concepts, and that web
becomes increasingly stable when when we can.
Speaker 3 (15:22):
Make the associations.
Speaker 2 (15:24):
More robust, more reliable, and then further when we can
invent things like science where we can really validate causal
relationships between things, and that makes our representations even more stable.
Speaker 1 (15:39):
I see, so human brains interact with one another and
work on how do we make these representations stable? How
do we get knowledge coming out like a Socratic dialectic,
but with with everybody all involved and so on. And
so your idea when you moved into this field of AGI,
howeveryone wants to define it? What was your idea?
Speaker 2 (16:01):
Well, so I left academia, which I absolutely loved, but
I felt an urgency to validate this theory.
Speaker 3 (16:11):
I don't I mean, it's just a theory at this point.
Speaker 2 (16:14):
How do we know whether we are optimized to align
our minds?
Speaker 3 (16:18):
There's so much evidence to suggest that we do.
Speaker 2 (16:20):
But as Richard Feyman said, you don't really know if
you understand something until you can build it.
Speaker 3 (16:24):
And so I thought, well, maybe I could build this thing.
And the moment is just right because AI is taking
off again.
Speaker 2 (16:31):
It's waking up from one of the winters, and I
was watching the scaling have really impressive results with a
deep learning, which felt really good because as somebody who
had a background in neuroscience, just like, oh yeah, inspiration
from brains is actually proving to be really effective. So
(16:53):
I moved into tech and started collaborating with the engineers
who were trying to build ever more capable intelligence. I'm
now in one of these frontier AGI labs and the
thing that we are going to be doing, which I
think is really differentiated from other approaches, is try to
build the communicative drive. Can we build agents that are
(17:15):
optimized for understanding each other's perspectives? And from that, can
we get emergent behaviors, emergent capabilities that we wouldn't get
from a single model on its own.
Speaker 1 (17:29):
So I just want to slow that down. So communicative drive,
that's the first time we've heard the term, So tell
us what that means.
Speaker 2 (17:36):
So communicative drive is the phrase that I use to
describe this compulsion that we have to align our minds
to establish representational alignment. You can imagine how the communicative
drive would interact with other dispositions that humans have. And importantly,
(17:57):
you have to think at the population level. So again
we have variation for every trait, and some of us
are more open, some of us are more closed to experience.
Speaker 3 (18:08):
But you can imagine. Okay, so in the case of
somebody who's really closed, Let's say that they are.
Speaker 2 (18:14):
In some communicative exchange and they detect a mismatch. So
somebody is clearly not understanding what they are saying.
Speaker 3 (18:27):
They have two choices. They can update.
Speaker 2 (18:31):
Their perspectives to the other person's, or they can try
to get the other person's perspective to look more like theirs.
What would it take to get another person to come
to your perspective.
Speaker 3 (18:43):
You'd have to create an artifact. You'd have to create.
Speaker 2 (18:46):
A word or a piece of art or a theory
to get them to really understand and take on your
perspective and.
Speaker 3 (18:54):
Close that gap.
Speaker 2 (18:55):
But if you're a very open person, if you're creative,
that might be your default. But if you're a little
bit more reserved, maybe you just take on the other
person's perspective.
Speaker 1 (19:07):
Is it that way or the other way? Sorry? If
I'm very open, I feel like I would take the
other person's perspective, so you.
Speaker 2 (19:12):
Can actually imagine both situations. Yes, So I score very
high on openness, and when I'm in communicative exchanges, I
often feel like, oh, wow, yeah, that's I've never thought
of it that way, or maybe I have, and I
want to add all these things and like it's a
very cooperative thing. I'm more thinking about the dynamics of
people who want to maintain tradition and status quo versus
(19:34):
people who want to challenge that. So oftentimes that maps
onto the dimension of openness. So if you see that
everybody around you seems to hold a different perspective than
you do, you're more likely to conform to their perspectives.
If you're somebody who might be a little bit more conservative,
(19:54):
not wanting to ruffle feathers that kind of thing.
Speaker 1 (19:57):
Well, I'm just trying to stand why I use the
word conservative there, because conservative meaning are like iinin exactly.
Oh but you're saying, maintain the group traditions. Yeah, okay,
got it.
Speaker 3 (20:09):
As opposed to being iconoclastic and innovative.
Speaker 1 (20:11):
I see, I see how you're using it. Okay, great.
So this is the idea is that people are always talking,
and depending on your personality type, what you're trying to
do is either align yourself with them or them with
you or whatever, or meet in the middle. But this,
this you feel, is the key to what human societies
bring as opposed to looking at individual brains. You know,
(20:32):
the history of neuroscience is all about looking at individual brains. Oh,
this is how the visual system works, as how decision
making works, how hearing works, whatever. But there's this new
feel that's been growing for the last twenty or thirty years,
which is called social neuroscience, which is all about, gosh,
we've got a lot of circuitry in our brains that
care about other brains. So this is the heart of
your interest. Is what happens when people are talking and aligning?
(20:55):
And why are we so driven to communicate instead of
let's imagine that you and I set down on a
bus next to each other, we'd probably chat as opposed
to just sit there and deal with our own brains. Okay,
so how does this map onto what you're interested in
doing in AI?
Speaker 2 (21:11):
Yes, so I am concerned about building something that resembles
our own intelligence, or something that resembles us because we
have all sorts of flaws and biases.
Speaker 3 (21:23):
The variability, I.
Speaker 2 (21:25):
Think is very useful, and we wouldn't be intelligent in
the way that we are without the variability. And you
might call some of that variability the bias, the unique
biases that we have.
Speaker 3 (21:34):
But I think if we try to reproduce.
Speaker 2 (21:36):
All of that, we're going to get a mirror of ourselves,
and that's not always the most effective way to augment
our intelligence. And I should back up and say, why
are we doing any of this? Why do we want
to build intelligence that looks like us. I think the
assumption that a lot of these the people, the engineers,
and these labs have is that, oh, of course it's
(21:57):
going to be extremely useful for us. It's going to
unlock this unprecedented era of human flourishing. But the assumption
that it's going to be really useful for us, I
think is taken for granted, and if you really think
about it, well, how because a lot of the examples
that we have from recent technology and algorithms is that
(22:19):
they actually take away our agency. We lose hours to scrolling,
we get stuck in echo chambers, we have autocomplete takeaway
our thinking, and we're starting to see.
Speaker 3 (22:31):
The same kinds of things with chatbots.
Speaker 2 (22:35):
We're also seeing that people are using these technologies and
very much augmenting their their own intelligence. I feel sometimes
like I'm having entirely new thoughts at an unprecedented pace
when I'm going back and forth, just like when you
were having amazing conversations with other people. We use each
other's minds as tools, but you can just do that
(22:56):
at a more rapid pace. So it's not a foregone
inclusion that giving the AI more capabilities, making it smarter
and giving it more agency is going to be good
for us. I think we have to turn that on
its head and say, what would it take to make
AI that makes us smarter and gives us more agency?
(23:19):
And that would be, by definition, something that is good
for us. So how do we do that? I don't
think that we want to have agents that have their
own drives to survive and manipulate us and have all
of the status seeking U situations that we have. But
(23:39):
if they were motivated to align their representations with ours,
that could actually be really useful for unlocking our potential
and for helping us learn. And as we're giving them
these capabilities to do that, they have to figure out.
One of the ways that we are able to generalize
and continually learn is that we are constantly negotiating meaning
(24:04):
and coming up and with the friction of the interactions
with each other, we are able to do continual learning
because we're not optimizing for one thing, one niche, one environment,
one particular problem. We are optimizing for aligning our minds
with many minds, and all of them. These targets are
(24:26):
all moving targets, so it's kind of an escape velocity
from really focusing on one thing and our ability to
do that not only allows us to continually learn, but
it gives us superpowers.
Speaker 1 (24:40):
So what does this look like for you?
Speaker 2 (24:42):
Though?
Speaker 1 (24:42):
If you had a world five years in the future
that you were able to sort of define where this
is going, what's ok Does it mean that there are
lots of AI agents and they are talking with humans
and they're trying to align their thinking with humans and
the humans a with the AI or its look like
there's one AI give us a sense of this world?
Speaker 2 (25:04):
Okay, So there's at least two important things here. One
is that right now agents are not reliable, so they're
not useful. And I think the idea there is that
they are fundamentally different than llms. They are embodied in
some kind of environment, even if it's the digital environment,
but we can't yet get them to do long horizon
you know, actions in a reliable way, and so they're
(25:27):
not yet useful.
Speaker 1 (25:28):
Right now, you're talking about AI agents.
Speaker 2 (25:30):
Most people have interacted with chatbots and that's what they
think AI is, or that's what they think generative AI is.
Maybe they know of you know, the image generators too,
but a lot of us are interacting with chatbots. Those
are llms that are predicting the next text token. Large
language models, yes, large language models, but they don't actually
do things. Agents, in contrast, can actually take actions and
(25:55):
do things on our behalf and in order.
Speaker 3 (25:59):
So are lab is.
Speaker 2 (26:00):
Working on building computer use agents. So if I want
an agent to book me a flight or order me
a dinner, I can say that and then it can
go off and use whatever websites or software tools to
do those things.
Speaker 3 (26:16):
That's the hope.
Speaker 2 (26:17):
You've seen a couple of these agents, computer use agents
come out, and it's really exciting to see them start
to do things, but they're not reliable. And because they're
not reliable, they might do the thing that you ask
them to do one out of ten times, and again
that's exciting, but that's not very useful, right.
Speaker 1 (26:35):
You mean it's because they make mistakes. It's not the
way we would say an employee is not reliable because
he's out back smoking a cigarette. Is that they're trying
to do stuff is just a clicking on the wrong
thing and getting it.
Speaker 2 (26:45):
Wrong, that's right y, Yes, So working on making these
agents reliable is necessary for making them useful. But we
can imagine that unlocking a whole new set of capabilities
and ways that they would augment humans because rather than
just having a conversation, and conversations can be very useful.
(27:06):
All of the things that you do in your daily life,
all the things that you're using a computer for, the
vast majority of them are probably not worthy of your time.
You're doing things on the computer to actually achieve other
things in the real world. So what if you could
have agents reliably execute a lot of the things that
you're doing.
Speaker 3 (27:25):
And in.
Speaker 2 (27:27):
Knowledge work, you know, we're using a ton of different
tools I call them arbitrary skills, processing invoices or something
that everybody does using doing their taxes.
Speaker 3 (27:37):
Like, do you really have to becommon expert.
Speaker 2 (27:39):
At using these tools or is that maybe not the
best use of our human potential, our cognitive potential. If
we could have agents that knew how to use all
of the tools that we did, that would save us
a ton of time, and it would have cascading implications
for how people collaborate with other people in the real world.
(28:00):
Human collaboration would be different because we'd be freed up
to focus on more creative things, more strategic decisions. Having
the sorts of debates that we have to advance whatever
shared goals that we have. So this is the sort
of first part. We have to just get the agents
to reliably click or scroll when we need them to.
(28:21):
But if you play that forward, what does reliability actually
mean when we have higher level goals.
Speaker 3 (28:29):
It's not just.
Speaker 2 (28:30):
Knowing where to click or knowing when to scroll. It's
actually understanding the goal. And that goal might require breaking
the breaking it down into subtasks, and then going and
doing all of those things, and there might be many
ways of doing it, and there's not necessarily a right
or a wrong way of doing it. So at the
(28:50):
end of the day, reliability ends up becoming about understanding
our minds.
Speaker 1 (29:10):
So the idea is if I could have an AI
agent that understands my mind, that has a model of me,
including what I know and don't know, and what my
goals are long term and short term, then it could
do a better job at what needs to be done.
Because when it comes to a choice point is is oh,
I know what Eagleman wants. He likes this kind of thing,
(29:31):
and that might be something that emerges not just from
patterns of looking at my behavior, but actually understanding internally,
having some theory of my mind.
Speaker 3 (29:39):
I think that it.
Speaker 2 (29:39):
Would need that, yes, and I think that we would
need to be able to interact with it in the
way that we interact with other teammates, where we're negotiating,
meaning in real time, where we're going back to earlier
in our conversation. Sometimes we don't realize that we're not
clearly thinking about something, and so having that reflected back
and being able to go through this exchange a dialectic.
(30:03):
It can refine our thinking, sharpen our thinking.
Speaker 1 (30:07):
So this is a thing I've been wondering about for
a while, which is if you're looking at something from
the outside, you can actually get a lot of data
about it. And by outside I mean as opposed to
from the inside. If I have a theory of your mind,
the question is if I, just if I could observe
all of your behavior without knowing anything about what's in
your mind, could I nonetheless do just as good a job.
Speaker 2 (30:30):
Well, I think that this is what our models of
other mind essentially are. It's making sense of behavior. Yeah,
it's just that the behavior, again is multiply redundant, and
there are many different cues that we can attend to,
and we're prioritizing some cues over others, and then it
is more efficient for us to represent that there's a
(30:52):
mind behind the eyes.
Speaker 1 (30:53):
Yeah, very good. Do you see a world where we
would have lots of AI agents that are also speeding
with one another in terms of communicative drive of saying, hey,
this is what I've learned, and I know and blah
blah blah, and they develop a better, something bigger as
a result of the communication. They have a better understanding
of the world because they're talking with one another, because
(31:14):
just like humans, each of them is going to have
some trajectory through space time.
Speaker 2 (31:19):
Okay, this gets into the second thing that I think
is really important about how agents can be useful. So
the first thing is they can do the digital drudgery
for us. They can save time. I call this They
can become our collective subconscious because we won't have to
spend our conscious time attending to things. We can relegate
so the agents can be doing in parallel all of
(31:41):
this stuff, so we don't have to become expert in
these arbitrary skills. But they are also as they are
doing that for a lot of people, they're learning a
bunch of different skills they're learning how to navigate different
websites and use different software tools. And so if I
need to for my job learn something really quickly, they
can redistribute the skills that they've learned from my teammates,
(32:02):
and they can give me the context that I need,
not to become expert in that tool, but to be
able to establish representational alignment with my teammate who uses
that tool. So they can help coordinate a team's behavior
by understanding all of the things that we do, all
the goals that we have, all the tools that we
use to achieve our goals.
Speaker 1 (32:22):
Now, normally that would happen. You go up to Susie
and you say, hey, you know, I need to talk
to you, and Susie says, no, I use a different
word than you're using.
Speaker 3 (32:29):
It for ten minutes to even establish common ground.
Speaker 1 (32:32):
Got it. But you're saying that AI could help with
that interaction like the third person in the room and say, hey,
you know what, Danielle, this is what you need to
know about and Susie, this is what you need to know.
And I noticed you guys are using this same word,
but you mean different things by it's that kind of thing.
Speaker 2 (32:45):
Yes, that's so yes to your point, I do think
that these agents that are working in parallel and understanding
our context will probably detect a ton of inefficiencies and
how we're doing things, and they will come up with
better ways of doing I would hope that they would.
Speaker 3 (33:01):
Do that great.
Speaker 2 (33:03):
One of the things that we learned is we were
trying to train our agent how to use Gmail, is
that wow, most people are actually really bad at using Gmail.
Speaker 1 (33:12):
In what way?
Speaker 3 (33:13):
So we don't know how to do.
Speaker 2 (33:15):
The search queries effectively. We mostly just stumble through. And
it's the power users who sometimes build hold businesses around
you know, the Google Suite and Gmail. They they know
exactly how it was designed, all of the affordances that
it has, all of the new features, how that can
make things actually more efficient.
Speaker 3 (33:36):
They get it.
Speaker 2 (33:37):
But most people don't have the time to keep up
with all of the new things that you can do,
and when they sit down to look at their email,
they just want to you know, send that email off,
or just want to find that thing, and so they're
not deeply engaged with learning all of the things you
can do.
Speaker 1 (33:54):
So if you had this AI agent sitting on the
shoulder in that sense, they would let's Sayea, they would
teach you. Hey, look, here's the thing you need to
know today, Daniel, is that the idea is that it
would help you to be a better Gmail user in
this particular example.
Speaker 2 (34:07):
Well, so, I actually think the longer term goal is
that humans are spending far.
Speaker 3 (34:11):
Less time looking at screens.
Speaker 1 (34:13):
Excellent.
Speaker 2 (34:15):
Yes, there are exceptions which include when the actual tool
scaffolds your thinking. So I use the example of Adobe
Creative Suite. If you want to edit a photo or
create a podcast or a video, if you didn't have
all of the UIs and all of the dropdowns, yes,
(34:38):
then you probably wouldn't even know where to start. You
wouldn't even know what was possible.
Speaker 3 (34:43):
So some of the tools.
Speaker 2 (34:45):
Are actually really helpful in scaffolding your understanding of what's possible,
whereas other tools just distract so much from the actual goal.
We have to learn how to use the tools to
do the actual research that we that we care about.
So having agents take over the things that we don't
care about is great, and then we can focus on
(35:06):
the interactions that really do matter, the visualizations that really
do matter.
Speaker 1 (35:10):
So let me just understand that. So you're saying, let's
say future software are ten years from now. I open it,
there's just a few simple things on the menu, but
there's lots of hidden power which my AI agent can
help me expose and uncover through time.
Speaker 2 (35:25):
Yes, and I'm imagining things where Okay, the agent understands
my goal, has my context and can generate on the
fly only the UI, the button or the search field
that I need in that moment. Every time I open
my computer, I feel anxious. I swipe to another tab
(35:45):
and it's like it's.
Speaker 3 (35:46):
Swiping my memory. What was I doing again? And that
is that is constant.
Speaker 2 (35:50):
You know, you've got most people have a lot of
tabs open, They've got a lot of tools open, and
it's just our cognition is not meant for all of that.
There's a lot of cognitive loads. So if we could
simplify that, that would augment our cognition. The other way
that agents could really augment our potential is by helping
us learn. So this kind of flows from it has
(36:14):
a model of my mind, my teammate's mind, It can
help us communicate at the right level of abstraction, save
us time. But also if it has a model of
my mind and it has my context, it knows the
things that I know and don't know, the skills that
I have, and the things that I care about. Then
say I want to learn something totally new. Maybe it's
not just a software tool. Maybe it's something like quantum mechanics,
(36:37):
And that's really difficult to understand. You need analogies, But
what are the right analogies that are going to work
for me? Well, if it has a model of my mind,
then it can, in a personalized way help me come
to the understanding that I need to get the big picture,
and it can sort of follow that in a way
(36:59):
create a curriculum for me that gives me the information
that's not too challenging, not too easy, that's right in
that sweet.
Speaker 1 (37:07):
Spot between frustrating and achievable. Yes, yeah, that's really interesting.
That'll have clear implications in education as well, keeping people
right at their right spot there. Okay, so let me
return to a question, because I just want to make
sure I understood coming back to this idea of communicative
drive and agents like you and I learning from one another.
(37:27):
Will AI agents learn from one another, not just between
agent and human, but agent to agent.
Speaker 2 (37:34):
Yes, And now you're really bringing it all together. So
I think that agents interacting with other agents will be
able to learn all sorts of patterns that maybe we
haven't yet learned, and detect all sorts of inefficiencies and
be more efficient in some ways.
Speaker 3 (37:47):
But if they also have a.
Speaker 2 (37:50):
Not only the ability to model our minds, but a
motivation to then that's not going to be restricted knowledge
to them. They're not going to go off and speciate
and have all of this you know, intell that we
don't have. They're going to try to communicate their insights
to us. So chess players are now so much better
after we've built AI that's really good at chess. So
we can co evolve with this new species of intelligence.
(38:14):
And if it's motivated to bring us along to establish
representational alignment, then I think we will continue to get smarter.
Speaker 1 (38:21):
Do you see a situation where representational alignment just isn't possible?
For example, let's say I came to you and said, hey, Danielle,
I really want to teach you about these really important
pieces of Mongolian history. And let's say you just don't
care about Mongolian history, and I'm trying to tell you
the state in this emperor or whatever. It's not going
to go very far.
Speaker 2 (38:40):
That is a really good question. I think you nailed
it when you said just don't care. In the same
way that it's really hard to teach a child something
that they don't care about, it's not relevant to them.
I think it will be hard to establish representational alignment
with anybody if they don't care. But in principle, I
think it's possible, given that there is ant It might
(39:01):
just be a matter of finding the right analogies. And
again this goes to your work. There's so much plasticity
in the brain.
Speaker 3 (39:10):
I think.
Speaker 2 (39:10):
Correct me if I'm wrong, But in principle, there's no
limit to what we could learn to not only understand,
but even have a phenomenological experience of if there is
structure to that information.
Speaker 1 (39:24):
Yes, but all of brain plasticity is driven by relevance.
In other words, do I care about it? I can't
think right. So if my AI agent comes to me
and says, look, I just realized this great thing about
how you could redesign this computer chip in this way,
and maybe it starts telling me all this detailed stuff
and I just don't care. It's not achieving representational alignments.
Speaker 2 (39:41):
That's true, And maybe in some cases, Okay, some people
don't care, other people do care.
Speaker 3 (39:45):
Go to the people who do care.
Speaker 2 (39:46):
But also I guess this also begs the question why
do some people care about things? Because it's relevant? So
make it relevant. Tell the person as an agent, lead
the person to the insights that they need to have
to about something. One of the things that is associated
with becoming expert in something is that you really like
the things that you become expert in because it is
(40:08):
satisfying to understand, and the more bits and pieces of
information that you can integrate into a holistic web of knowledge,
the better it feels.
Speaker 3 (40:17):
So I think that this is my vision for a
very happy future, is that.
Speaker 2 (40:25):
We're all learning all the time, and the more we
learn and the more sort of accurate our shared world
models map onto reality, the more satisfying it will be
to continue to learn.
Speaker 1 (40:51):
I totally agree, and I love thinking about this from
the point of view of education, because if you think
of this sphere of humankind's knowledge, we know more than
any of us could possibly learn in a lifetime. Yeah,
So the key is to find out what are the
doors on the outside of this sphere that you love
or that I love? Given our totally different backgrounds and whatever.
(41:11):
Certain things really fascinate me and other things I wish
but they don't. Okay. So if I can enter this
door and you enter this other door and we end
up learning the same kind of stuff, it's really great. Yeah.
And Isaac Asimov actually really cared about this topic way
back in the day, and he did an interview on LERR,
which was this PBS talk show thing way back in
(41:32):
the eighties, and he envisioned this was before the Internet,
and he said, I envisioned a day where there's a
huge supercomputer that has all of human kinds of knowledge,
and everyone has a cable running from this computer to
their home and you can ask the computer any question
you want. But his point was you could take your
own inroad into this sphere of knowledge. Okay. So your
(41:55):
point is if these AI agents have some theory about
our minds, as in your mind, in my mind, everybody
is an individual, and then can cast things in a
certain way like look, here's a way that you might
care about it, then we're all going to learn faster
and better.
Speaker 3 (42:12):
Yeah.
Speaker 2 (42:13):
Yeah, And I also think that this is not just
you know, academic stuff, intellectual stuff.
Speaker 3 (42:18):
Anything can be interesting.
Speaker 2 (42:20):
You know, people in their knitting communities can become really
passionate about innovating new ways and in sharing that.
Speaker 3 (42:27):
Within a community.
Speaker 2 (42:28):
And I think that this also gets to one of
the sort of essences of what we care about as humans.
It's this idea of optimal distinctiveness. So we simultaneously need
communities for belonging.
Speaker 3 (42:42):
This is an evolutionarily ancient thing.
Speaker 2 (42:44):
It's not just humans, but we need to feel like
we belong within a community.
Speaker 3 (42:47):
We've got our in group, we've got our tribe.
Speaker 2 (42:49):
Not always bad, and maybe it's just something that we
will always need as being human, but we also need
to feel like we're unique and that we have something
to contribute to the community. We are optimally distinct in
our contributions. So I imagine a future where agents with
models of our minds really allow us to be diverse.
(43:13):
They're not flattening the experiences or the capabilities, but they're
encouraging diversity and variability. But they're also building mechanisms for
us to align our minds. They're building bridges throughout the
various experiences and capabilities so great.
Speaker 1 (43:32):
So this leads me to this question that I've been
wondering about which is. You know, in political debates that
we see lots nowadays, there isn't a representational alignment. If
you know someone's on the left and someone's on the right,
they end up saying we're not going to have Thanksgiving
dinner together instead of saying, oh, let me understand. How
can I understand? So where does representational alignment break down?
(43:54):
And might AI help us there someday?
Speaker 2 (43:57):
I think representational alignment breaks down when two minds have
such different starting points, such different sets of maybe analogies, experiences,
and also motivations, things.
Speaker 3 (44:13):
That they care about, that it's just really hard.
Speaker 2 (44:16):
And let's assume that they can get past the initial
emotional friction of just knowing that they come from different groups.
Speaker 3 (44:24):
That's not trivial.
Speaker 2 (44:25):
Sometimes just knowing that somebody is from a different group
prevents any sort of establishing of common ground. But assuming
that you can get over that, then you just might
not have enough overlap in your representations. I suspect that
that's very rare. Just by nature of being a human
embodied in the way that we are having experiences in
(44:47):
the world and the way that we do, there's gonna
be enough overlap as an entry point into being able
to establish some kind of alignment.
Speaker 1 (44:57):
So sorry, you're saying it's rare that people would have
such different pints view that can't get there, because it
is the case that people don't get there or won't
get there. But you're saying if people tried harder, let's say,
with political arguments. Yes.
Speaker 2 (45:09):
I also think that as we interact with this hypothetical
agent of the future that has models of our minds,
it will be modeling for us behavior that helps us
establish alignment. And so in the same way that we
learn from others who model good behavior and then we
(45:29):
start to reflect that subconsciously, I think it could nudge
us into more pro social interaction.
Speaker 1 (45:37):
I totally agree with this. I did a podcast episode
on this about this AI research group in Europe that
released some chat bots onto a Reddit channel that does debate,
and they didn't tell anybody that these were aibots, so
they got in big trouble. Everyone was mad about it.
But what happened was these AI bots would come in
and take someone who had a particular point of view,
(45:57):
and they would take the other point of view and
they would discuss, and on this particular channel, you get
points if you successfully convince somebody of your point of view.
And so it turns out these bots did six times
better than humans do on average in terms of changing
the other person's mind. So everyone freaked out about this
and said, oh my god, these AI debate bots can
(46:18):
manipulate us. And but it turns out the really amazing
part is they weren't doing anything manipulative. They weren't lying,
they weren't doing anything. They were just better debaters in
the sense that they were empathic, they were calm, they
presented their arguments well, And I thought, God, we can
really learn from that if we teach our children to
be better discussings.
Speaker 3 (46:41):
Totally.
Speaker 1 (46:41):
Yeah, So I love that. I love that point. But
what you're saying is, so you think people with differentints
of view can do representational alignment. But that's a separate issue,
which is that how do we do things culturally and
teaching them to be better at conversation.
Speaker 2 (46:56):
Yeah, and even having an awareness that other people are different.
Speaker 3 (47:00):
It's not personal.
Speaker 2 (47:01):
If you disagree or if you have different rituals, different
ways of doing things, that's fine. So I think even
just exposure to variability diversity can get us part of
the way there.
Speaker 1 (47:12):
Oh, that's fascinating. Cool. The other thing I was going
to ask you about is you mentioned earlier just tangentially
said something about archaeology.
Speaker 2 (47:19):
Ah, yes, okay, So there over the past two decades,
there's been an update in our understanding of the evolution
of the types of sophisticated reasoning and thinking symbolic thinking
capabilities that humans have.
Speaker 3 (47:35):
We used to think.
Speaker 2 (47:36):
That they emerged suddenly in what we call the cognitive
revolution that happened thirty to forty thousand years ago, and
that's because.
Speaker 1 (47:44):
The so humans were like other primates and then suddenly,
thirty four thousand years ago something happened.
Speaker 2 (47:49):
Okay, yeah, and archaeologists were looking at evidence in Europe
and the caves and you know, the art and things
like that, and it just it seemed like there was
a discontinuity. But then they started exploring throughout Africa and
using more nuanced methods, and a lot more of them
(48:10):
started doing this, and they started finding evidence from as
early as three hundred thousand years ago that we were
cognitively modern. But what seems to be the important thing
was population density and contact with other groups. So the
idea is that our sophisticated cognitive capabilities are latent until
(48:34):
we come into contact with each other, until we poke
each other's brains. And you see that this evidence ebbs
and flows, It appears and disappears as a function of
these group donaities.
Speaker 1 (48:48):
Yeah, oh fascinating. Okay, So when humans come into contact
with other humans, but not just their own tribe, presumably
other tribes, bigger and bigger civilizations, slightly.
Speaker 3 (48:57):
Different ways of making their tools and their weapons and
their jewelry.
Speaker 1 (49:01):
Yes, oh excellent, Oh that's beautiful. So, by the way,
is it thought that there was some discontinuity where that
became possible. In other words, if you stick a bunch
of capuchin monkeys together.
Speaker 3 (49:15):
They will never they'll never get there, right.
Speaker 1 (49:17):
Right, So there's so there's something different.
Speaker 2 (49:20):
And I mean, I think it's this stability of our
the models of our minds, and I think it's this
communicative drive. But you need a critical density of people
and you also need the variability.
Speaker 1 (49:31):
Okay, so let's just summarize it. So it's having theory
of mind in other words, knowing Okay, Danielle has her
own thoughts, her own representations in there, and then you
add that to the density of people.
Speaker 3 (49:42):
And the variability of different groups coming together ability.
Speaker 1 (49:45):
That's excellent.
Speaker 3 (49:45):
Yeah.
Speaker 2 (49:46):
So the takeaway from this archaeological evidence is that becoming
cognitively modern was this slow, gradual process over the course
of the last couple hundred thousand years.
Speaker 1 (49:58):
But it was predicated on pop density yeah, and people
coming together.
Speaker 2 (50:02):
Yeah.
Speaker 1 (50:02):
Oh excellent. Oh I love that. That's so interesting, And it.
Speaker 2 (50:06):
Goes along with this idea that we're not just biologically evolving,
we're culturally evolving. And cultural evolution does a lot of
the work in explaining human behavior.
Speaker 1 (50:19):
Yeah, and biological evolution, of course, is super slow, but
cultural evolution is so rapid.
Speaker 3 (50:24):
Yes.
Speaker 2 (50:25):
And actually I think that agents that have models of
our minds can help reconcile some of the tensions that
we're seeing because cultural evolution is outpacing biological evolution. So
what happens when you've succeeded in society, you've done well
in education, you go to the workplace and you're supposed to,
you know, contribute your intelligence. You end up staring at
(50:49):
a screen for most of the day and not interacting
with other people. The infrastructure actually doesn't really support unlocking
our potential because of all of the sort of arbitrary
things that have happened culturally. We've built these incredible devices
and we've co evolved with them, and now they're extensions
of our intelligence. But they're also we're also conforming to
(51:11):
them rather than the other way around.
Speaker 1 (51:13):
True, But they are social, as in social media and
so on. I mean, when I'm staring at my screen,
I'm interacting with thousands of people in various ways, whether
I'm looking at extra Instagram or I'm doing emails. In
a sense, it's more social than humans ever could have been.
Speaker 2 (51:28):
What this is the problem when the algorithms are not
aligned with our well being, with our potential. So I
see there are so many mistakes that we've made over
the past ten fifteen years that we can learn from
and hopefully not repeat with more capable AI.
Speaker 1 (51:45):
But out of curiosity, if I'm on X and I
see that there are different points of view about this
political thing that I happen to care about, then I'm
getting exposed to lots of points of view.
Speaker 2 (51:54):
Right.
Speaker 3 (51:54):
Well, I'm not saying it's all bad. Certainly, Yes, I think.
Speaker 2 (51:57):
That having online communities is absolutely a step in the
right direction for connecting us. It's fantastic but the way
that they are optimized and the attention economy not serving
us but serving advertisers is a problem.
Speaker 1 (52:12):
But what would you change about let's say something like X.
What would you change about social media to make it
so it's more optimized?
Speaker 3 (52:20):
Well, I think optimization is the problem.
Speaker 1 (52:23):
So sorry, I met more optimized for a communicative drive.
Speaker 2 (52:26):
I wouldn't think of it that way because I think
that you have the agents, but then you also have
how they are dynamically interacting with each other. And X
is or any social media platform, is one narrow way
of facilitating interactions. In the case of X, it's very
short form blurbs, and it appeals to the fact of
(52:50):
human nature that we are more interested in things that
have shock value and things that are negative or disgusting,
and how do you work against that. It's not about
informational exchange, although some people lean more in terms of
caring about that, and so you do see some of that.
Speaker 1 (53:09):
Yeah, so that's interesting. So we are in the sense
of population density for three hundred thousand year old tribes.
It is the case that you see on X all
kinds of points of view that you didn't know existed.
And well, I don't know if you do.
Speaker 2 (53:24):
Because you're in your echo chamber, you typically tend to
not be exposed to a lot of variability.
Speaker 1 (53:28):
Well, you know what's interesting, what you're exposed to is
the most extreme views of the other side because people
in your echo chamber say, look at what this idiot
is on the other side of the aisle said.
Speaker 3 (53:39):
And the polarize it.
Speaker 1 (53:41):
Yeah exactly, Yeah, okay, So wrapping up for today, the
key thing is that intelligence is not just about what's
happening in an individual brain, but it's social.
Speaker 2 (53:53):
Yes, this is true about all intelligences. All intelligences emerge
as a function of their environments and interacting with their
environments the problems within the environments that the organisms have
to solve. The most challenging problems in humans environments are
understanding other humans because we have to figure out how
(54:13):
to cooperate, We have to figure out.
Speaker 3 (54:15):
How to align our minds.
Speaker 2 (54:18):
So our intelligence emerges from our interactions with each other,
and we continually ratchet up our intelligence by co evolving
with each other.
Speaker 1 (54:32):
That was my conversation with Danielle Persik. One of the
main threads was that maybe we shouldn't be thinking of
intelligence as something that's packaged up inside a single head.
Another way to look at it is as something that
emerges through interaction, through the friction, through the shared effort
of understanding another mind. Human intelligence has always been shaped
(54:55):
by this social dimension. You can see this from the
way that infants learn about the world to the way
that societies build knowledge over generations. What today's conversation invites
us to reconsider is the idea that learning and understanding,
and fundamentally alignment are really the central features of intelligence.
(55:19):
Our ability to model other minds, to recognize that other
people see the world differently, that they know different things,
that they care about different things. This is what allows
cooperation and culture and cumulative progress. Through this lens, intelligence
is about negotiating meaning. Now, if we take on that lens,
(55:43):
the future of AI looks very different from what most
people are thinking about now, because this shifts the conversation
away from only asking how capable AI systems are going
to be. Now we're pushed to ask how they participate
in our cognitive ecosystems. In other words, in Danielle's view,
(56:06):
how can we develop agents who help us think better
by reducing friction and clarifying misunderstandings between people and supporting
learning at the right level of abstraction, instead of merely
replacing us, which is the doomsayers version of the future.
(56:27):
Could AI agents serve as mediators and translators and collaborators.
And there's another issue I found fascinating. Alignment is something
humans have always learned through interactions. So perhaps instead of
just viewing AI alignment as a technical problem to be solved,
(56:48):
we could see it as a behavior to be modeled.
Systems that reflect our values back to us might teach
us how to communicate more effectively with one another, even
in moments of disagreement. If you're a regular listener to
this podcast, you know that I'm obsessed with issues about
the brain and polarization, and so this possibility that AI
(57:11):
might actually be able to mediate between us and help
us get our curiosity back that feels especially consequential. So
as AI agents become more embedded in our daily life,
the choices that we make now about their design are
going to shape how we relate to them and to
(57:34):
our fellow humans. The question is whether they will help
intelligence continue to sprout in new directions. In other words,
maybe will be even more intelligent as a species thanks
to our machines. Go to eagleman dot com slash podcast
(57:58):
for more information and to find further reading. Join the
weekly discussions on my substack, and check out and subscribe
to Inner Cosmos on YouTube for videos of each episode
and to leave comments. Until next time, I'm David Eagleman,
and this is Inner Cosmos.
Is it possible that we're thinking about intelligence in the
wrong way? Instead of being something inside individual brains, is
intelligence instead something that emerges from lots of brains that
are constantly working to align with one another. And if
we take on that lens, what does this mean about
(00:26):
the way that we can build AI agents or the
way that they can make us better? What is the
difference between information and information with a purpose? Today we're
going to speak with Daniel Persick, a cognitive scientist who
leads the Human Computer Interaction team at Amazon's AGI Lab.
(00:47):
So get ready for a great brain stretch. Welcome to
Intercosmos with me David Eagleman. I'm a neuroscientist and an
author at Stanford and in these episodes, as we sail
deeply into our three pound universe to understand how we
see the world and soon how AI might come to
(01:08):
understand the world with us. Let's think about the word intelligence.
You might justifiably assume that neuroscientists have an agreed upon
(01:30):
definition for this, but we actually don't. However one thinks
about intelligence, I think it's a fair assumption that most
of us, when we think about it, assume that intelligence
is something that happens inside a single head, in other words,
a brain processing information. This statement seems so obvious that
(01:52):
it hardly invites inspection, but if you step back and
look at how intelligence actually unfolds in a human life,
a different picture can start to emerge. Our thinking is
shaped by other people from the very beginning. We learn
by watching, by imitating, by trying to communicate, and eventually
(02:15):
by negotiating meaning with the people around us. Even our
most private thoughts are built from tools that are fundamentally social,
things like language and symbols and shared concepts and cultural norms.
So this may sound strange, but this is what we're
going to talk about today, and the idea will become
very clear. Most of humanity's greatest achievements didn't come from
(02:39):
lone geniuses working in isolation, but from really dense networks
of minds interacting over time. When we look at things
like science or art, or morality or technology, it almost
never makes sense to interpret these as products of individual intelligence,
but instead they are collective processes that allow ideas to
(03:04):
collide and to form into something and to continuously evolve.
So intelligence in this sense may be less like a
thing we possess and more like something that emerges between us. Now,
this broader perspective becomes especially important as we find ourselves
(03:26):
flinging headlong into the era of artificial intelligence. With every
passing week, we're getting AI acting more like a participant.
We're getting systems that communicate but also agents that act
on our behalf to do things in the world. And
soon these agents will collaborate with each other at their
(03:49):
time scales and spatial scales. So if intelligence is social
by nature, then building the future world of AI might
end up requiring more than just dumping billions into scaling
up the training data for these systems. It may instead
require understanding how minds relate to one another in the
(04:13):
first place. And that's where today's conversation begins. Today I'm
joined by Danielle Persk. She's a cognitive scientist who leads
the Human Computer Interaction team at Amazon's AGI Lab. Danielle
uses insights from the evolution and development of human intelligence
to inform how we can not only make AI smarter,
(04:36):
but build AI that also makes us smarter. Here's my
conversation with Danielle Persk.
Speaker 2 (04:47):
Intelligence in humans is really social, and that is the
thing that differentiates our intelligence from other species. Even other
species that are closely related to us have similar brain
structures and function similar genetics. And what we are really
optimizing for is representing other minds. So not only are
(05:10):
infants human infants inferring the existence of other minds, but
once this thing exists, we are optimized for aligning our minds. Evolutionarily,
we had to cooperate to survive. Infants need to be
able to have their caretaker's attention on them to survive,
and in terms of being able to learn about the world,
(05:33):
once infants have a model of other minds, then they
can manipulate it. They can direct their caretaker's attention point
what's that, and magically they'll have a label for.
Speaker 3 (05:41):
This thing that they're looking at in their environment.
Speaker 1 (05:43):
So they're doing prompt engineering.
Speaker 3 (05:47):
Great technology. Yeah, okay, so we know that.
Speaker 2 (05:50):
You know, throughout the course of human evolution, we became
increasingly dependent upon cooperating to to stay alive and adapt
to new environment. So it makes sense that there'd be
this extreme pressure on being able to predict each other's
behaviors to understand our minds, and then with infants, developmentally,
we have also the benefit of being able to learn
(06:13):
much more efficiently even language itself, from representing other minds.
Speaker 1 (06:19):
Okay, so it turns out that we can do a
much better job of predicting if we can imagine what
it's like to be inside other people's heads. Right, So,
if I want to know what some non player character
is going to do in a video game whatever, they
have certain behaviors. But if I want to know, let's
say what you're going to do next, or say next,
(06:39):
if I have a model of your mind and what
you know and you don't know and all that stuff,
I can make a better prediction.
Speaker 2 (06:45):
And so you've said that there's information and then information
with a purpose, and that information with a purpose really matters.
Speaker 3 (06:53):
So you've used the example of like the.
Speaker 2 (06:56):
Land rover on Mars not being able to fix itself,
and like a wolf that gets its like trap.
Speaker 1 (07:03):
Right, the Curiosity Rover went up to Mars. We had
spent like a billion something dollars on it. It did
a great job on Mars, but eventually it got its
right front wheel stuck in the Martian soil and it
died couldn't get out. But if you can trast that
with a wolf who gets its leg cond of trap.
It'll chew its leg off and then figure out how
to walk on three legs, which is extraordinary because a
(07:24):
wolf's brain didn't evolve for three legs. But it can
figure it out because it's live wired. It has brain
plasticity and figure out, Okay, how do I adjust everything.
Speaker 3 (07:33):
So that I can survival?
Speaker 1 (07:34):
Depends upon it exactly. That's the key. It has relevance
to the animal.
Speaker 2 (07:39):
Right, So all animals have a drive to survive, a
drive to reproduce, But humans also have a drive to
align our minds because it helps us cooperate, it helps
us survive, and it helps us to learn extremely efficiently.
So we don't just model other minds. That would just
(07:59):
be the information part. We are optimized for aligning our minds.
So it's information with a purpose.
Speaker 1 (08:05):
Okay, so aligning our minds this is the key thing
and at the center of your interests. And so then
you went into looking into AGI. So first of all,
tell us what artificial general intelligence is to you.
Speaker 2 (08:18):
Well, I think most of the labs that are trying
to build something like AGI, they all have their own definitions.
None of them are really very good. But the one
thing that unifies all of them is that they are
all benchmarked to human intelligence. And this goes all the
way back to the origin of the field of AI.
(08:39):
So in nineteen fifty six, a group of these engineers
and mathematicians got together. They were going to solve intelligence
and build thinking machines, and the idea is that these
thinking machines would think like us.
Speaker 3 (08:49):
It obviously took a.
Speaker 2 (08:50):
Very long time to realize, Oh, that's a lot harder
than we thought that it was. But now we are
back to aiming for something like that original goal of
building thinking machines that think like us.
Speaker 3 (09:01):
We call it AGI.
Speaker 2 (09:03):
Again, have slightly different operationalizations. But I think that we're
all running towards the wrong thing. And that's because I
don't think that intelligence can exist in a machine. It
doesn't exist in individual humans. It's something that emerges from
our interactions because we have this drive to align our representations,
(09:27):
and of course we all have very different representations.
Speaker 3 (09:30):
Right When I used to teach.
Speaker 2 (09:31):
Cognitive science, I would teach about this condition called a fantasia,
and once every couple of classes a student would come
up to.
Speaker 1 (09:40):
Me quick In fantations where you can't imagine, you don't
have any visual representation on the Yes.
Speaker 2 (09:45):
Yes, a student would come up to me and say, wait,
you mean there are people who can actually imagine things.
Speaker 3 (09:51):
Their mind's eye is not just a metaphor. It's a
thing that.
Speaker 2 (09:53):
People experience, and they wouldn't know because they don't suffer
from other types of death. It's just one of the
many ways in which human cognition and experience can very
And when I imagine in apple, it's different than when
you imagine an apple. We all have different associations. So
(10:15):
when we come together and we have to use words
to try to align our minds, there's necessarily going to
be friction, especially when we're trying to talk about abstract things,
especially when we're talking about things at the bleeding edge
of our knowledge, like science.
Speaker 3 (10:34):
How do you align.
Speaker 2 (10:37):
Your representations when there's not even a word for something.
So intelligence emerges as a function of trying to align
our minds and oftentimes creating new concepts to achieve that.
Speaker 1 (10:51):
Okay, so when you're talking about aligning minds, it's because
I've got my whole internal world. You've got your whole
internal world that is built by each of our our
trajectories through space time. We've had different experiences all these things.
So we come together and we've got completely different worlds
running on the inside. And that's what conversation is about.
We're trying to align things that way.
Speaker 2 (11:12):
And there are neuroscientists who measure when people are either
communicating in real time or if they're listening to a story,
if they're watching something on a screen, you can measure
the degree of neuralsynchrony, how close they are to be
on the same wavelength, and that predicts all sorts of
things like how much they like each other, how much
(11:32):
they understood the story, and how much they liked the story,
how similarly they remember things.
Speaker 1 (11:39):
Okay, so this is what humans do. We get together
in conversation all the time and we try to achieve
that synchrony in terms of oh, okay, wait, you have
a different view than I do on this, here's how
we can make progress. This is the Socratic dialectic, right.
This is what Socrates love to do, is have these
conversations where the truth emerges, something bigger than either person
(12:00):
knew when they started the conversation.
Speaker 2 (12:03):
And on that point too, I think a lot of
us think that we know things, but actually when we're
forced to describe something we realized we don't.
Speaker 1 (12:14):
Yeah. Actually, in my next book, I'm talking about this
as a Potempkin village. Yeah, so you know. The Potemkin village,
for anyone doesn't remember, is when it was Catherine the
Great of Russia was heading down the river with a
bunch of dignitaries that she was trying to impress. She
hired this skuy Potempkin. Actually he was her lover as
well as a military general, but she got him to
(12:34):
go down the river a long way and build what
looked like a facade of a village so that when
the ship went by, all the dignitaries would be impressed
that there was this village. And he got all these
peasants like walk around happily and stuff, But there were
no buildings. It was just the front face of the building.
And then when the ship passed, he deconstructed this and
(12:57):
went ahead and built another village so that they passed
another great village, so it looked like things were really
happening there. Anyway, Cognition is often like this, where we think, oh, yeah,
I got it. Here's an example that I often use
is for anybody listening, take out a piece of paper,
and draw a bicycle, draw a bike.
Speaker 3 (13:16):
I've tried this, yeah hard, Yeah.
Speaker 1 (13:19):
Exactly, it turns out, I mean something as simple as
a bike what you see every day. Yeah, you start realizing, wait,
actually I don't know exactly where this goes and what's
the thing and so anyway, Yes, this is an example
of where we think we have deep knowledge and sometimes
it's just the facade of something that we know.
Speaker 2 (13:35):
Yeah, and you can apply it on all different levels.
So you're describing, like the visual imagery might not be
very stable, but a lot of concepts are not stable either,
and we invent ways of making them more stable. Words
are a classic example of that. Once you have a
word for something, you can more easily trigger it, you
(13:57):
can more easily remember it, you can use it and
manipulate it and apply it to different things. But there's
a whole class of things that we're constantly inventing to
better align our minds. They're called cognitive technologies. So writing
would be one of the original ones. But symbols like math, logic, Yeah.
Speaker 1 (14:17):
So unpack that. What's an example of this?
Speaker 3 (14:19):
So literally, any word that you learn. Let's go back
to apples.
Speaker 2 (14:23):
So children see apples, they don't have a word associated
with it, so the likelihood that it's going to spontaneously
sort of emerge in their mind is very low. Maybe
if they've seen a couple then there will be some
sort of increased likelihood or lowered threshold. But once they
(14:44):
have a word for that thing, that they reliably associate
it with it, then anybody who says the word anytime
they hear it, now their brain will elicit that activity
and it becomes a more stable representation. That's an obvious
example where where there's actually a physical thing that the
word can refer to.
Speaker 3 (15:03):
But what about.
Speaker 2 (15:04):
Concepts like love and justice that you can't see you're
saying By assigning a word to it, then we make
that stable, and then it become it can become associated
with a whole web of other concepts, and that web
becomes increasingly stable when when we can.
Speaker 3 (15:22):
Make the associations.
Speaker 2 (15:24):
More robust, more reliable, and then further when we can
invent things like science where we can really validate causal
relationships between things, and that makes our representations even more stable.
Speaker 1 (15:39):
I see, so human brains interact with one another and
work on how do we make these representations stable? How
do we get knowledge coming out like a Socratic dialectic,
but with with everybody all involved and so on. And
so your idea when you moved into this field of AGI,
howeveryone wants to define it? What was your idea?
Speaker 2 (16:01):
Well, so I left academia, which I absolutely loved, but
I felt an urgency to validate this theory.
Speaker 3 (16:11):
I don't I mean, it's just a theory at this point.
Speaker 2 (16:14):
How do we know whether we are optimized to align
our minds?
Speaker 3 (16:18):
There's so much evidence to suggest that we do.
Speaker 2 (16:20):
But as Richard Feyman said, you don't really know if
you understand something until you can build it.
Speaker 3 (16:24):
And so I thought, well, maybe I could build this thing.
And the moment is just right because AI is taking
off again.
Speaker 2 (16:31):
It's waking up from one of the winters, and I
was watching the scaling have really impressive results with a
deep learning, which felt really good because as somebody who
had a background in neuroscience, just like, oh yeah, inspiration
from brains is actually proving to be really effective. So
(16:53):
I moved into tech and started collaborating with the engineers
who were trying to build ever more capable intelligence. I'm
now in one of these frontier AGI labs and the
thing that we are going to be doing, which I
think is really differentiated from other approaches, is try to
build the communicative drive. Can we build agents that are
(17:15):
optimized for understanding each other's perspectives? And from that, can
we get emergent behaviors, emergent capabilities that we wouldn't get
from a single model on its own.
Speaker 1 (17:29):
So I just want to slow that down. So communicative drive,
that's the first time we've heard the term, So tell
us what that means.
Speaker 2 (17:36):
So communicative drive is the phrase that I use to
describe this compulsion that we have to align our minds
to establish representational alignment. You can imagine how the communicative
drive would interact with other dispositions that humans have. And importantly,
(17:57):
you have to think at the population level. So again
we have variation for every trait, and some of us
are more open, some of us are more closed to experience.
Speaker 3 (18:08):
But you can imagine. Okay, so in the case of
somebody who's really closed, Let's say that they are.
Speaker 2 (18:14):
In some communicative exchange and they detect a mismatch. So
somebody is clearly not understanding what they are saying.
Speaker 3 (18:27):
They have two choices. They can update.
Speaker 2 (18:31):
Their perspectives to the other person's, or they can try
to get the other person's perspective to look more like theirs.
What would it take to get another person to come
to your perspective.
Speaker 3 (18:43):
You'd have to create an artifact. You'd have to create.
Speaker 2 (18:46):
A word or a piece of art or a theory
to get them to really understand and take on your
perspective and.
Speaker 3 (18:54):
Close that gap.
Speaker 2 (18:55):
But if you're a very open person, if you're creative,
that might be your default. But if you're a little
bit more reserved, maybe you just take on the other
person's perspective.
Speaker 1 (19:07):
Is it that way or the other way? Sorry? If
I'm very open, I feel like I would take the
other person's perspective, so you.
Speaker 2 (19:12):
Can actually imagine both situations. Yes, So I score very
high on openness, and when I'm in communicative exchanges, I
often feel like, oh, wow, yeah, that's I've never thought
of it that way, or maybe I have, and I
want to add all these things and like it's a
very cooperative thing. I'm more thinking about the dynamics of
people who want to maintain tradition and status quo versus
(19:34):
people who want to challenge that. So oftentimes that maps
onto the dimension of openness. So if you see that
everybody around you seems to hold a different perspective than
you do, you're more likely to conform to their perspectives.
If you're somebody who might be a little bit more conservative,
(19:54):
not wanting to ruffle feathers that kind of thing.
Speaker 1 (19:57):
Well, I'm just trying to stand why I use the
word conservative there, because conservative meaning are like iinin exactly.
Oh but you're saying, maintain the group traditions. Yeah, okay,
got it.
Speaker 3 (20:09):
As opposed to being iconoclastic and innovative.
Speaker 1 (20:11):
I see, I see how you're using it. Okay, great.
So this is the idea is that people are always talking,
and depending on your personality type, what you're trying to
do is either align yourself with them or them with
you or whatever, or meet in the middle. But this,
this you feel, is the key to what human societies
bring as opposed to looking at individual brains. You know,
(20:32):
the history of neuroscience is all about looking at individual brains. Oh,
this is how the visual system works, as how decision
making works, how hearing works, whatever. But there's this new
feel that's been growing for the last twenty or thirty years,
which is called social neuroscience, which is all about, gosh,
we've got a lot of circuitry in our brains that
care about other brains. So this is the heart of
your interest. Is what happens when people are talking and aligning?
(20:55):
And why are we so driven to communicate instead of
let's imagine that you and I set down on a
bus next to each other, we'd probably chat as opposed
to just sit there and deal with our own brains. Okay,
so how does this map onto what you're interested in
doing in AI?
Speaker 2 (21:11):
Yes, so I am concerned about building something that resembles
our own intelligence, or something that resembles us because we
have all sorts of flaws and biases.
Speaker 3 (21:23):
The variability, I.
Speaker 2 (21:25):
Think is very useful, and we wouldn't be intelligent in
the way that we are without the variability. And you
might call some of that variability the bias, the unique
biases that we have.
Speaker 3 (21:34):
But I think if we try to reproduce.
Speaker 2 (21:36):
All of that, we're going to get a mirror of ourselves,
and that's not always the most effective way to augment
our intelligence. And I should back up and say, why
are we doing any of this? Why do we want
to build intelligence that looks like us. I think the
assumption that a lot of these the people, the engineers,
and these labs have is that, oh, of course it's
(21:57):
going to be extremely useful for us. It's going to
unlock this unprecedented era of human flourishing. But the assumption
that it's going to be really useful for us, I
think is taken for granted, and if you really think
about it, well, how because a lot of the examples
that we have from recent technology and algorithms is that
(22:19):
they actually take away our agency. We lose hours to scrolling,
we get stuck in echo chambers, we have autocomplete takeaway
our thinking, and we're starting to see.
Speaker 3 (22:31):
The same kinds of things with chatbots.
Speaker 2 (22:35):
We're also seeing that people are using these technologies and
very much augmenting their their own intelligence. I feel sometimes
like I'm having entirely new thoughts at an unprecedented pace
when I'm going back and forth, just like when you
were having amazing conversations with other people. We use each
other's minds as tools, but you can just do that
(22:56):
at a more rapid pace. So it's not a foregone
inclusion that giving the AI more capabilities, making it smarter
and giving it more agency is going to be good
for us. I think we have to turn that on
its head and say, what would it take to make
AI that makes us smarter and gives us more agency?
(23:19):
And that would be, by definition, something that is good
for us. So how do we do that? I don't
think that we want to have agents that have their
own drives to survive and manipulate us and have all
of the status seeking U situations that we have. But
(23:39):
if they were motivated to align their representations with ours,
that could actually be really useful for unlocking our potential
and for helping us learn. And as we're giving them
these capabilities to do that, they have to figure out.
One of the ways that we are able to generalize
and continually learn is that we are constantly negotiating meaning
(24:04):
and coming up and with the friction of the interactions
with each other, we are able to do continual learning
because we're not optimizing for one thing, one niche, one environment,
one particular problem. We are optimizing for aligning our minds
with many minds, and all of them. These targets are
(24:26):
all moving targets, so it's kind of an escape velocity
from really focusing on one thing and our ability to
do that not only allows us to continually learn, but
it gives us superpowers.
Speaker 1 (24:40):
So what does this look like for you?
Speaker 2 (24:42):
Though?
Speaker 1 (24:42):
If you had a world five years in the future
that you were able to sort of define where this
is going, what's ok Does it mean that there are
lots of AI agents and they are talking with humans
and they're trying to align their thinking with humans and
the humans a with the AI or its look like
there's one AI give us a sense of this world?
Speaker 2 (25:04):
Okay, So there's at least two important things here. One
is that right now agents are not reliable, so they're
not useful. And I think the idea there is that
they are fundamentally different than llms. They are embodied in
some kind of environment, even if it's the digital environment,
but we can't yet get them to do long horizon
you know, actions in a reliable way, and so they're
(25:27):
not yet useful.
Speaker 1 (25:28):
Right now, you're talking about AI agents.
Speaker 2 (25:30):
Most people have interacted with chatbots and that's what they
think AI is, or that's what they think generative AI is.
Maybe they know of you know, the image generators too,
but a lot of us are interacting with chatbots. Those
are llms that are predicting the next text token. Large
language models, yes, large language models, but they don't actually
do things. Agents, in contrast, can actually take actions and
(25:55):
do things on our behalf and in order.
Speaker 3 (25:59):
So are lab is.
Speaker 2 (26:00):
Working on building computer use agents. So if I want
an agent to book me a flight or order me
a dinner, I can say that and then it can
go off and use whatever websites or software tools to
do those things.
Speaker 3 (26:16):
That's the hope.
Speaker 2 (26:17):
You've seen a couple of these agents, computer use agents
come out, and it's really exciting to see them start
to do things, but they're not reliable. And because they're
not reliable, they might do the thing that you ask
them to do one out of ten times, and again
that's exciting, but that's not very useful, right.
Speaker 1 (26:35):
You mean it's because they make mistakes. It's not the
way we would say an employee is not reliable because
he's out back smoking a cigarette. Is that they're trying
to do stuff is just a clicking on the wrong
thing and getting it.
Speaker 2 (26:45):
Wrong, that's right y, Yes, So working on making these
agents reliable is necessary for making them useful. But we
can imagine that unlocking a whole new set of capabilities
and ways that they would augment humans because rather than
just having a conversation, and conversations can be very useful.
(27:06):
All of the things that you do in your daily life,
all the things that you're using a computer for, the
vast majority of them are probably not worthy of your time.
You're doing things on the computer to actually achieve other
things in the real world. So what if you could
have agents reliably execute a lot of the things that
you're doing.
Speaker 3 (27:25):
And in.
Speaker 2 (27:27):
Knowledge work, you know, we're using a ton of different
tools I call them arbitrary skills, processing invoices or something
that everybody does using doing their taxes.
Speaker 3 (27:37):
Like, do you really have to becommon expert.
Speaker 2 (27:39):
At using these tools or is that maybe not the
best use of our human potential, our cognitive potential. If
we could have agents that knew how to use all
of the tools that we did, that would save us
a ton of time, and it would have cascading implications
for how people collaborate with other people in the real world.
(28:00):
Human collaboration would be different because we'd be freed up
to focus on more creative things, more strategic decisions. Having
the sorts of debates that we have to advance whatever
shared goals that we have. So this is the sort
of first part. We have to just get the agents
to reliably click or scroll when we need them to.
(28:21):
But if you play that forward, what does reliability actually
mean when we have higher level goals.
Speaker 3 (28:29):
It's not just.
Speaker 2 (28:30):
Knowing where to click or knowing when to scroll. It's
actually understanding the goal. And that goal might require breaking
the breaking it down into subtasks, and then going and
doing all of those things, and there might be many
ways of doing it, and there's not necessarily a right
or a wrong way of doing it. So at the
(28:50):
end of the day, reliability ends up becoming about understanding
our minds.
Speaker 1 (29:10):
So the idea is if I could have an AI
agent that understands my mind, that has a model of me,
including what I know and don't know, and what my
goals are long term and short term, then it could
do a better job at what needs to be done.
Because when it comes to a choice point is is oh,
I know what Eagleman wants. He likes this kind of thing,
(29:31):
and that might be something that emerges not just from
patterns of looking at my behavior, but actually understanding internally,
having some theory of my mind.
Speaker 3 (29:39):
I think that it.
Speaker 2 (29:39):
Would need that, yes, and I think that we would
need to be able to interact with it in the
way that we interact with other teammates, where we're negotiating,
meaning in real time, where we're going back to earlier
in our conversation. Sometimes we don't realize that we're not
clearly thinking about something, and so having that reflected back
and being able to go through this exchange a dialectic.
(30:03):
It can refine our thinking, sharpen our thinking.
Speaker 1 (30:07):
So this is a thing I've been wondering about for
a while, which is if you're looking at something from
the outside, you can actually get a lot of data
about it. And by outside I mean as opposed to
from the inside. If I have a theory of your mind,
the question is if I, just if I could observe
all of your behavior without knowing anything about what's in
your mind, could I nonetheless do just as good a job.
Speaker 2 (30:30):
Well, I think that this is what our models of
other mind essentially are. It's making sense of behavior. Yeah,
it's just that the behavior, again is multiply redundant, and
there are many different cues that we can attend to,
and we're prioritizing some cues over others, and then it
is more efficient for us to represent that there's a
(30:52):
mind behind the eyes.
Speaker 1 (30:53):
Yeah, very good. Do you see a world where we
would have lots of AI agents that are also speeding
with one another in terms of communicative drive of saying, hey,
this is what I've learned, and I know and blah
blah blah, and they develop a better, something bigger as
a result of the communication. They have a better understanding
of the world because they're talking with one another, because
(31:14):
just like humans, each of them is going to have
some trajectory through space time.
Speaker 2 (31:19):
Okay, this gets into the second thing that I think
is really important about how agents can be useful. So
the first thing is they can do the digital drudgery
for us. They can save time. I call this They
can become our collective subconscious because we won't have to
spend our conscious time attending to things. We can relegate
so the agents can be doing in parallel all of
(31:41):
this stuff, so we don't have to become expert in
these arbitrary skills. But they are also as they are
doing that for a lot of people, they're learning a
bunch of different skills they're learning how to navigate different
websites and use different software tools. And so if I
need to for my job learn something really quickly, they
can redistribute the skills that they've learned from my teammates,
(32:02):
and they can give me the context that I need,
not to become expert in that tool, but to be
able to establish representational alignment with my teammate who uses
that tool. So they can help coordinate a team's behavior
by understanding all of the things that we do, all
the goals that we have, all the tools that we
use to achieve our goals.
Speaker 1 (32:22):
Now, normally that would happen. You go up to Susie
and you say, hey, you know, I need to talk
to you, and Susie says, no, I use a different
word than you're using.
Speaker 3 (32:29):
It for ten minutes to even establish common ground.
Speaker 1 (32:32):
Got it. But you're saying that AI could help with
that interaction like the third person in the room and say, hey,
you know what, Danielle, this is what you need to
know about and Susie, this is what you need to know.
And I noticed you guys are using this same word,
but you mean different things by it's that kind of thing.
Speaker 2 (32:45):
Yes, that's so yes to your point, I do think
that these agents that are working in parallel and understanding
our context will probably detect a ton of inefficiencies and
how we're doing things, and they will come up with
better ways of doing I would hope that they would.
Speaker 3 (33:01):
Do that great.
Speaker 2 (33:03):
One of the things that we learned is we were
trying to train our agent how to use Gmail, is
that wow, most people are actually really bad at using Gmail.
Speaker 1 (33:12):
In what way?
Speaker 3 (33:13):
So we don't know how to do.
Speaker 2 (33:15):
The search queries effectively. We mostly just stumble through. And
it's the power users who sometimes build hold businesses around
you know, the Google Suite and Gmail. They they know
exactly how it was designed, all of the affordances that
it has, all of the new features, how that can
make things actually more efficient.
Speaker 3 (33:36):
They get it.
Speaker 2 (33:37):
But most people don't have the time to keep up
with all of the new things that you can do,
and when they sit down to look at their email,
they just want to you know, send that email off,
or just want to find that thing, and so they're
not deeply engaged with learning all of the things you
can do.
Speaker 1 (33:54):
So if you had this AI agent sitting on the
shoulder in that sense, they would let's Sayea, they would
teach you. Hey, look, here's the thing you need to
know today, Daniel, is that the idea is that it
would help you to be a better Gmail user in
this particular example.
Speaker 2 (34:07):
Well, so, I actually think the longer term goal is
that humans are spending far.
Speaker 3 (34:11):
Less time looking at screens.
Speaker 1 (34:13):
Excellent.
Speaker 2 (34:15):
Yes, there are exceptions which include when the actual tool
scaffolds your thinking. So I use the example of Adobe
Creative Suite. If you want to edit a photo or
create a podcast or a video, if you didn't have
all of the UIs and all of the dropdowns, yes,
(34:38):
then you probably wouldn't even know where to start. You
wouldn't even know what was possible.
Speaker 3 (34:43):
So some of the tools.
Speaker 2 (34:45):
Are actually really helpful in scaffolding your understanding of what's possible,
whereas other tools just distract so much from the actual goal.
We have to learn how to use the tools to
do the actual research that we that we care about.
So having agents take over the things that we don't
care about is great, and then we can focus on
(35:06):
the interactions that really do matter, the visualizations that really
do matter.
Speaker 1 (35:10):
So let me just understand that. So you're saying, let's
say future software are ten years from now. I open it,
there's just a few simple things on the menu, but
there's lots of hidden power which my AI agent can
help me expose and uncover through time.
Speaker 2 (35:25):
Yes, and I'm imagining things where Okay, the agent understands
my goal, has my context and can generate on the
fly only the UI, the button or the search field
that I need in that moment. Every time I open
my computer, I feel anxious. I swipe to another tab
(35:45):
and it's like it's.
Speaker 3 (35:46):
Swiping my memory. What was I doing again? And that
is that is constant.
Speaker 2 (35:50):
You know, you've got most people have a lot of
tabs open, They've got a lot of tools open, and
it's just our cognition is not meant for all of that.
There's a lot of cognitive loads. So if we could
simplify that, that would augment our cognition. The other way
that agents could really augment our potential is by helping
us learn. So this kind of flows from it has
(36:14):
a model of my mind, my teammate's mind, It can
help us communicate at the right level of abstraction, save
us time. But also if it has a model of
my mind and it has my context, it knows the
things that I know and don't know, the skills that
I have, and the things that I care about. Then
say I want to learn something totally new. Maybe it's
not just a software tool. Maybe it's something like quantum mechanics,
(36:37):
And that's really difficult to understand. You need analogies, But
what are the right analogies that are going to work
for me? Well, if it has a model of my mind,
then it can, in a personalized way help me come
to the understanding that I need to get the big picture,
and it can sort of follow that in a way
(36:59):
create a curriculum for me that gives me the information
that's not too challenging, not too easy, that's right in
that sweet.
Speaker 1 (37:07):
Spot between frustrating and achievable. Yes, yeah, that's really interesting.
That'll have clear implications in education as well, keeping people
right at their right spot there. Okay, so let me
return to a question, because I just want to make
sure I understood coming back to this idea of communicative
drive and agents like you and I learning from one another.
(37:27):
Will AI agents learn from one another, not just between
agent and human, but agent to agent.
Speaker 2 (37:34):
Yes, And now you're really bringing it all together. So
I think that agents interacting with other agents will be
able to learn all sorts of patterns that maybe we
haven't yet learned, and detect all sorts of inefficiencies and
be more efficient in some ways.
Speaker 3 (37:47):
But if they also have a.
Speaker 2 (37:50):
Not only the ability to model our minds, but a
motivation to then that's not going to be restricted knowledge
to them. They're not going to go off and speciate
and have all of this you know, intell that we
don't have. They're going to try to communicate their insights
to us. So chess players are now so much better
after we've built AI that's really good at chess. So
we can co evolve with this new species of intelligence.
(38:14):
And if it's motivated to bring us along to establish
representational alignment, then I think we will continue to get smarter.
Speaker 1 (38:21):
Do you see a situation where representational alignment just isn't possible?
For example, let's say I came to you and said, hey, Danielle,
I really want to teach you about these really important
pieces of Mongolian history. And let's say you just don't
care about Mongolian history, and I'm trying to tell you
the state in this emperor or whatever. It's not going
to go very far.
Speaker 2 (38:40):
That is a really good question. I think you nailed
it when you said just don't care. In the same
way that it's really hard to teach a child something
that they don't care about, it's not relevant to them.
I think it will be hard to establish representational alignment
with anybody if they don't care. But in principle, I
think it's possible, given that there is ant It might
(39:01):
just be a matter of finding the right analogies. And
again this goes to your work. There's so much plasticity
in the brain.
Speaker 3 (39:10):
I think.
Speaker 2 (39:10):
Correct me if I'm wrong, But in principle, there's no
limit to what we could learn to not only understand,
but even have a phenomenological experience of if there is
structure to that information.
Speaker 1 (39:24):
Yes, but all of brain plasticity is driven by relevance.
In other words, do I care about it? I can't
think right. So if my AI agent comes to me
and says, look, I just realized this great thing about
how you could redesign this computer chip in this way,
and maybe it starts telling me all this detailed stuff
and I just don't care. It's not achieving representational alignments.
Speaker 2 (39:41):
That's true, And maybe in some cases, Okay, some people
don't care, other people do care.
Speaker 3 (39:45):
Go to the people who do care.
Speaker 2 (39:46):
But also I guess this also begs the question why
do some people care about things? Because it's relevant? So
make it relevant. Tell the person as an agent, lead
the person to the insights that they need to have
to about something. One of the things that is associated
with becoming expert in something is that you really like
the things that you become expert in because it is
(40:08):
satisfying to understand, and the more bits and pieces of
information that you can integrate into a holistic web of knowledge,
the better it feels.
Speaker 3 (40:17):
So I think that this is my vision for a
very happy future, is that.
Speaker 2 (40:25):
We're all learning all the time, and the more we
learn and the more sort of accurate our shared world
models map onto reality, the more satisfying it will be
to continue to learn.
Speaker 1 (40:51):
I totally agree, and I love thinking about this from
the point of view of education, because if you think
of this sphere of humankind's knowledge, we know more than
any of us could possibly learn in a lifetime. Yeah,
So the key is to find out what are the
doors on the outside of this sphere that you love
or that I love? Given our totally different backgrounds and whatever.
(41:11):
Certain things really fascinate me and other things I wish
but they don't. Okay. So if I can enter this
door and you enter this other door and we end
up learning the same kind of stuff, it's really great. Yeah.
And Isaac Asimov actually really cared about this topic way
back in the day, and he did an interview on LERR,
which was this PBS talk show thing way back in
(41:32):
the eighties, and he envisioned this was before the Internet,
and he said, I envisioned a day where there's a
huge supercomputer that has all of human kinds of knowledge,
and everyone has a cable running from this computer to
their home and you can ask the computer any question
you want. But his point was you could take your
own inroad into this sphere of knowledge. Okay. So your
(41:55):
point is if these AI agents have some theory about
our minds, as in your mind, in my mind, everybody
is an individual, and then can cast things in a
certain way like look, here's a way that you might
care about it, then we're all going to learn faster
and better.
Speaker 3 (42:12):
Yeah.
Speaker 2 (42:13):
Yeah, And I also think that this is not just
you know, academic stuff, intellectual stuff.
Speaker 3 (42:18):
Anything can be interesting.
Speaker 2 (42:20):
You know, people in their knitting communities can become really
passionate about innovating new ways and in sharing that.
Speaker 3 (42:27):
Within a community.
Speaker 2 (42:28):
And I think that this also gets to one of
the sort of essences of what we care about as humans.
It's this idea of optimal distinctiveness. So we simultaneously need
communities for belonging.
Speaker 3 (42:42):
This is an evolutionarily ancient thing.
Speaker 2 (42:44):
It's not just humans, but we need to feel like
we belong within a community.
Speaker 3 (42:47):
We've got our in group, we've got our tribe.
Speaker 2 (42:49):
Not always bad, and maybe it's just something that we
will always need as being human, but we also need
to feel like we're unique and that we have something
to contribute to the community. We are optimally distinct in
our contributions. So I imagine a future where agents with
models of our minds really allow us to be diverse.
(43:13):
They're not flattening the experiences or the capabilities, but they're
encouraging diversity and variability. But they're also building mechanisms for
us to align our minds. They're building bridges throughout the
various experiences and capabilities so great.
Speaker 1 (43:32):
So this leads me to this question that I've been
wondering about which is. You know, in political debates that
we see lots nowadays, there isn't a representational alignment. If
you know someone's on the left and someone's on the right,
they end up saying we're not going to have Thanksgiving
dinner together instead of saying, oh, let me understand. How
can I understand? So where does representational alignment break down?
(43:54):
And might AI help us there someday?
Speaker 2 (43:57):
I think representational alignment breaks down when two minds have
such different starting points, such different sets of maybe analogies, experiences,
and also motivations, things.
Speaker 3 (44:13):
That they care about, that it's just really hard.
Speaker 2 (44:16):
And let's assume that they can get past the initial
emotional friction of just knowing that they come from different groups.
Speaker 3 (44:24):
That's not trivial.
Speaker 2 (44:25):
Sometimes just knowing that somebody is from a different group
prevents any sort of establishing of common ground. But assuming
that you can get over that, then you just might
not have enough overlap in your representations. I suspect that
that's very rare. Just by nature of being a human
embodied in the way that we are having experiences in
(44:47):
the world and the way that we do, there's gonna
be enough overlap as an entry point into being able
to establish some kind of alignment.
Speaker 1 (44:57):
So sorry, you're saying it's rare that people would have
such different pints view that can't get there, because it
is the case that people don't get there or won't
get there. But you're saying if people tried harder, let's say,
with political arguments. Yes.
Speaker 2 (45:09):
I also think that as we interact with this hypothetical
agent of the future that has models of our minds,
it will be modeling for us behavior that helps us
establish alignment. And so in the same way that we
learn from others who model good behavior and then we
(45:29):
start to reflect that subconsciously, I think it could nudge
us into more pro social interaction.
Speaker 1 (45:37):
I totally agree with this. I did a podcast episode
on this about this AI research group in Europe that
released some chat bots onto a Reddit channel that does debate,
and they didn't tell anybody that these were aibots, so
they got in big trouble. Everyone was mad about it.
But what happened was these AI bots would come in
and take someone who had a particular point of view,
(45:57):
and they would take the other point of view and
they would discuss, and on this particular channel, you get
points if you successfully convince somebody of your point of view.
And so it turns out these bots did six times
better than humans do on average in terms of changing
the other person's mind. So everyone freaked out about this
and said, oh my god, these AI debate bots can
(46:18):
manipulate us. And but it turns out the really amazing
part is they weren't doing anything manipulative. They weren't lying,
they weren't doing anything. They were just better debaters in
the sense that they were empathic, they were calm, they
presented their arguments well, And I thought, God, we can
really learn from that if we teach our children to
be better discussings.
Speaker 3 (46:41):
Totally.
Speaker 1 (46:41):
Yeah, So I love that. I love that point. But
what you're saying is, so you think people with differentints
of view can do representational alignment. But that's a separate issue,
which is that how do we do things culturally and
teaching them to be better at conversation.
Speaker 2 (46:56):
Yeah, and even having an awareness that other people are different.
Speaker 3 (47:00):
It's not personal.
Speaker 2 (47:01):
If you disagree or if you have different rituals, different
ways of doing things, that's fine. So I think even
just exposure to variability diversity can get us part of
the way there.
Speaker 1 (47:12):
Oh, that's fascinating. Cool. The other thing I was going
to ask you about is you mentioned earlier just tangentially
said something about archaeology.
Speaker 2 (47:19):
Ah, yes, okay, So there over the past two decades,
there's been an update in our understanding of the evolution
of the types of sophisticated reasoning and thinking symbolic thinking
capabilities that humans have.
Speaker 3 (47:35):
We used to think.
Speaker 2 (47:36):
That they emerged suddenly in what we call the cognitive
revolution that happened thirty to forty thousand years ago, and
that's because.
Speaker 1 (47:44):
The so humans were like other primates and then suddenly,
thirty four thousand years ago something happened.
Speaker 2 (47:49):
Okay, yeah, and archaeologists were looking at evidence in Europe
and the caves and you know, the art and things
like that, and it just it seemed like there was
a discontinuity. But then they started exploring throughout Africa and
using more nuanced methods, and a lot more of them
(48:10):
started doing this, and they started finding evidence from as
early as three hundred thousand years ago that we were
cognitively modern. But what seems to be the important thing
was population density and contact with other groups. So the
idea is that our sophisticated cognitive capabilities are latent until
(48:34):
we come into contact with each other, until we poke
each other's brains. And you see that this evidence ebbs
and flows, It appears and disappears as a function of
these group donaities.
Speaker 1 (48:48):
Yeah, oh fascinating. Okay, So when humans come into contact
with other humans, but not just their own tribe, presumably
other tribes, bigger and bigger civilizations, slightly.
Speaker 3 (48:57):
Different ways of making their tools and their weapons and
their jewelry.
Speaker 1 (49:01):
Yes, oh excellent, Oh that's beautiful. So, by the way,
is it thought that there was some discontinuity where that
became possible. In other words, if you stick a bunch
of capuchin monkeys together.
Speaker 3 (49:15):
They will never they'll never get there, right.
Speaker 1 (49:17):
Right, So there's so there's something different.
Speaker 2 (49:20):
And I mean, I think it's this stability of our
the models of our minds, and I think it's this
communicative drive. But you need a critical density of people
and you also need the variability.
Speaker 1 (49:31):
Okay, so let's just summarize it. So it's having theory
of mind in other words, knowing Okay, Danielle has her
own thoughts, her own representations in there, and then you
add that to the density of people.
Speaker 3 (49:42):
And the variability of different groups coming together ability.
Speaker 1 (49:45):
That's excellent.
Speaker 3 (49:45):
Yeah.
Speaker 2 (49:46):
So the takeaway from this archaeological evidence is that becoming
cognitively modern was this slow, gradual process over the course
of the last couple hundred thousand years.
Speaker 1 (49:58):
But it was predicated on pop density yeah, and people
coming together.
Speaker 2 (50:02):
Yeah.
Speaker 1 (50:02):
Oh excellent. Oh I love that. That's so interesting, And it.
Speaker 2 (50:06):
Goes along with this idea that we're not just biologically evolving,
we're culturally evolving. And cultural evolution does a lot of
the work in explaining human behavior.
Speaker 1 (50:19):
Yeah, and biological evolution, of course, is super slow, but
cultural evolution is so rapid.
Speaker 3 (50:24):
Yes.
Speaker 2 (50:25):
And actually I think that agents that have models of
our minds can help reconcile some of the tensions that
we're seeing because cultural evolution is outpacing biological evolution. So
what happens when you've succeeded in society, you've done well
in education, you go to the workplace and you're supposed to,
you know, contribute your intelligence. You end up staring at
(50:49):
a screen for most of the day and not interacting
with other people. The infrastructure actually doesn't really support unlocking
our potential because of all of the sort of arbitrary
things that have happened culturally. We've built these incredible devices
and we've co evolved with them, and now they're extensions
of our intelligence. But they're also we're also conforming to
(51:11):
them rather than the other way around.
Speaker 1 (51:13):
True, But they are social, as in social media and
so on. I mean, when I'm staring at my screen,
I'm interacting with thousands of people in various ways, whether
I'm looking at extra Instagram or I'm doing emails. In
a sense, it's more social than humans ever could have been.
Speaker 2 (51:28):
What this is the problem when the algorithms are not
aligned with our well being, with our potential. So I
see there are so many mistakes that we've made over
the past ten fifteen years that we can learn from
and hopefully not repeat with more capable AI.
Speaker 1 (51:45):
But out of curiosity, if I'm on X and I
see that there are different points of view about this
political thing that I happen to care about, then I'm
getting exposed to lots of points of view.
Speaker 2 (51:54):
Right.
Speaker 3 (51:54):
Well, I'm not saying it's all bad. Certainly, Yes, I think.
Speaker 2 (51:57):
That having online communities is absolutely a step in the
right direction for connecting us. It's fantastic but the way
that they are optimized and the attention economy not serving
us but serving advertisers is a problem.
Speaker 1 (52:12):
But what would you change about let's say something like X.
What would you change about social media to make it
so it's more optimized?
Speaker 3 (52:20):
Well, I think optimization is the problem.
Speaker 1 (52:23):
So sorry, I met more optimized for a communicative drive.
Speaker 2 (52:26):
I wouldn't think of it that way because I think
that you have the agents, but then you also have
how they are dynamically interacting with each other. And X
is or any social media platform, is one narrow way
of facilitating interactions. In the case of X, it's very
short form blurbs, and it appeals to the fact of
(52:50):
human nature that we are more interested in things that
have shock value and things that are negative or disgusting,
and how do you work against that. It's not about
informational exchange, although some people lean more in terms of
caring about that, and so you do see some of that.
Speaker 1 (53:09):
Yeah, so that's interesting. So we are in the sense
of population density for three hundred thousand year old tribes.
It is the case that you see on X all
kinds of points of view that you didn't know existed.
And well, I don't know if you do.
Speaker 2 (53:24):
Because you're in your echo chamber, you typically tend to
not be exposed to a lot of variability.
Speaker 1 (53:28):
Well, you know what's interesting, what you're exposed to is
the most extreme views of the other side because people
in your echo chamber say, look at what this idiot
is on the other side of the aisle said.
Speaker 3 (53:39):
And the polarize it.
Speaker 1 (53:41):
Yeah exactly, Yeah, okay, So wrapping up for today, the
key thing is that intelligence is not just about what's
happening in an individual brain, but it's social.
Speaker 2 (53:53):
Yes, this is true about all intelligences. All intelligences emerge
as a function of their environments and interacting with their
environments the problems within the environments that the organisms have
to solve. The most challenging problems in humans environments are
understanding other humans because we have to figure out how
(54:13):
to cooperate, We have to figure out.
Speaker 3 (54:15):
How to align our minds.
Speaker 2 (54:18):
So our intelligence emerges from our interactions with each other,
and we continually ratchet up our intelligence by co evolving
with each other.
Speaker 1 (54:32):
That was my conversation with Danielle Persik. One of the
main threads was that maybe we shouldn't be thinking of
intelligence as something that's packaged up inside a single head.
Another way to look at it is as something that
emerges through interaction, through the friction, through the shared effort
of understanding another mind. Human intelligence has always been shaped
(54:55):
by this social dimension. You can see this from the
way that infants learn about the world to the way
that societies build knowledge over generations. What today's conversation invites
us to reconsider is the idea that learning and understanding,
and fundamentally alignment are really the central features of intelligence.
(55:19):
Our ability to model other minds, to recognize that other
people see the world differently, that they know different things,
that they care about different things. This is what allows
cooperation and culture and cumulative progress. Through this lens, intelligence
is about negotiating meaning. Now, if we take on that lens,
(55:43):
the future of AI looks very different from what most
people are thinking about now, because this shifts the conversation
away from only asking how capable AI systems are going
to be. Now we're pushed to ask how they participate
in our cognitive ecosystems. In other words, in Danielle's view,
(56:06):
how can we develop agents who help us think better
by reducing friction and clarifying misunderstandings between people and supporting
learning at the right level of abstraction, instead of merely
replacing us, which is the doomsayers version of the future.
(56:27):
Could AI agents serve as mediators and translators and collaborators.
And there's another issue I found fascinating. Alignment is something
humans have always learned through interactions. So perhaps instead of
just viewing AI alignment as a technical problem to be solved,
(56:48):
we could see it as a behavior to be modeled.
Systems that reflect our values back to us might teach
us how to communicate more effectively with one another, even
in moments of disagreement. If you're a regular listener to
this podcast, you know that I'm obsessed with issues about
the brain and polarization, and so this possibility that AI
(57:11):
might actually be able to mediate between us and help
us get our curiosity back that feels especially consequential. So
as AI agents become more embedded in our daily life,
the choices that we make now about their design are
going to shape how we relate to them and to
(57:34):
our fellow humans. The question is whether they will help
intelligence continue to sprout in new directions. In other words,
maybe will be even more intelligent as a species thanks
to our machines. Go to eagleman dot com slash podcast
(57:58):
for more information and to find further reading. Join the
weekly discussions on my substack, and check out and subscribe
to Inner Cosmos on YouTube for videos of each episode
and to leave comments. Until next time, I'm David Eagleman,
and this is Inner Cosmos.
Host
David Eagleman
Popular Podcasts
Dateline NBC
Current and classic episodes, featuring compelling true-crime mysteries, powerful documentaries and in-depth investigations. Follow now to get the latest episodes of Dateline NBC completely free, or subscribe to Dateline Premium for ad-free listening and exclusive bonus content: DatelinePremium.com
The Breakfast Club
The World's Most Dangerous Morning Show, The Breakfast Club, With DJ Envy, Jess Hilarious, And Charlamagne Tha God!
Stuff You Should Know
If you've ever wanted to know about champagne, satanism, the Stonewall Uprising, chaos theory, LSD, El Nino, true crime and Rosa Parks, then look no further. Josh and Chuck have you covered.