May 7, 2026 • 33 mins
Tristan Harris seemed almost destined for the Silicon Valley life. He was raised in the Bay Area, studied computer science at Stanford, went to school with the people who made Instagram, and got his startup acquired by Google. But being inside the Silicon Valley machine exposed him to its inner workings, namely incentives that benefit the company and its bottom line, not the humans using the products.
Tristan’s non-profit, The Center for Humane Technology, wants to create a world where technology works in the service of people, that fosters human growth and connection. He has spent years raising the red flag about social media, which, he says, is made to hijack our attention. And now he’s turned his attention to AI, which he says is trying to hijack our attachment, our intimacy and replace our closest relationships. In this conversation, host Laurie Segall and Tristan talk about the power and responsibility of AI, what we can learn from the social media attention race, how we regulate technology that moves so fast, and how tech can actually become a net-benefit for humans.
If you have thoughts or questions for Laurie about this episode or anything Mostly Human, email us at hello@mostlyhuman.com
See omnystudio.com/listener for privacy information.
Listen
Watch
Mark as Played
Transcript
Episode Transcript
Available transcripts are automatically generated. Complete accuracy is not guaranteed.
Speaker 1 (00:00):
I think the key principle is is the technology being
designed to replace teachers or help make teachers better teachers.
He is the technology being designed to replace your relationships
or deep in your ability to have better human relationships.
Speaker 2 (00:14):
Tristan Harris loves technology. He wants to live in a
world where tech is in the service of people. It
fosters human growth and connection. Unfortunately, right now that's not
exactly what tech is doing, and Tristan and is nonprofit
the Center for Humane Technology are trying to help. Tristan's
background is really interesting. He's a Bay Area kid educated
(00:34):
at Stanford with all the other tech leaders. His friends
in college actually started Instagram. He launched his own startup
that was acquired by Google, where he went and worked
for a number of years. So Tristan knows and has
known the handful of people who are designing and building
the technology we use every day, technology like Instagram and
other social media apps that were all now addicted to
(00:57):
Through the Center for Humane Technology, Tristan has been raising
the red flag about social media, pointing out the many
ways that these platforms exploit human psychology and vulnerability, how
they can isolate us and make us dependent on the
doom scroll. It's a model that has led to what
Tristan calls human downgrading, where the tech gets better, but
(01:17):
it changes our lives for the worse, and it's a
model that AI is now following. Tristan is concerned that
AI has potential to be the next social media, only
this go around, instead of hacking our attention, AI is
hacking our attachment. I'm Lori Siegel, and you're listening to
Mostly Human, a tech podcast through a human lens. Tristan,
(01:47):
I've known you throughout the years, and you've always been
the person who's speaking very eloquently about what's coming around
the corners and what we're not talking about.
Speaker 3 (01:55):
So let's go to this moment.
Speaker 2 (01:57):
You were talking about social media for the last like
fifteen years in the harms if we're not careful, But
now we're in an AI moment, and this moment I
almost feel like, and you can correct me if I'm wrong,
the stakes are even higher ground us in this new reality.
Speaker 1 (02:12):
So you know, probably when you and I first started
talking about maybe it, first introduced to each other in
twenty thirteen, how could we predict so much of the
effects that what social media would do a more addicted, distracted, polarized,
sexualized culture. We could predict all of that because of
one thing is if you look at the incentives, meaning
(02:34):
that people say, well, social media, it's giving people a voice,
helps people connect with your friends. Yeah, it does those things.
But is Facebook's business model, is TikTok's business model improving
the health of society or giving people a voice? Or
is their business model showing whatever at your nervous system
keeps you scrolling in a doom scrolling loop. And obviously
it's the latter, And so I think one of the
(02:55):
key things we have to do is demystify which direction
technologies going by getting clear about what the incentives are.
So if you flash to AI, there's all these things
it could do. Why are we seeing the major AI
companies release these AI deep fake slop apps meaning it's
like a TikTok, but it's just AI generated deep fake
(03:16):
slopped So Meta released Vibes, Open Ai released Sora, And
why are they doing that when they said they're here
to cure cancer and they're here to solve climate change
and they're releasing something that just keeps people scrolling. And
the answer is because they're racing for market dominance. They're
racing to get to AGI, they're racing to get to
basically owning the world economy first. And what that's important
(03:39):
about getting that is that we are not going to
get this utopian world on the current trajectory because they're
racing to release the most powerful, inscrutable, uncontrollable technology we've
ever invented with the worst possible incentives of getting there first,
rather than making sure we do those things right, Will.
Speaker 3 (03:58):
You talk about the incentives right?
Speaker 2 (03:59):
You talk about the incentives of this new generation with
artificial intelligence, SAM incentives for social media?
Speaker 1 (04:07):
Well, yeah, there's so there's different things here. So what
was behind social media was the attention economy. There's a
finite supply of human attention that companies are in an
arms race to harvest bigger and bigger slices of it.
When they do that, they have to aggressively push out
the other guy by going deeper down into your brain stem,
hacking social psychology, hacking fear of missing out, hacking, social validation.
(04:27):
And that's what got us the catastrophe that we're now
living in. And so when you watch one huge societal
catastrophe unfold through a kind of blind spot of not
facing the consequences of what's really at stake. Having gone
through that experience, it's like, I don't want to see
that it happening again.
Speaker 2 (04:47):
Yeah with Ai, I think we both probably feel this
so personally because I think about you talk about like
you went to college with these folks. I think about
being a young journalist twenty three years old in the
CNN newsroom and being like, there's this really cool happening.
Speaker 3 (05:01):
It's called tech.
Speaker 2 (05:02):
And I remember always thinking like and asking these human
questions of like, oh, like that sounds amazing.
Speaker 3 (05:07):
But have you thought about this?
Speaker 2 (05:08):
And I think this is why this moment is really
personal to both of us, because you your background was
at Google and in design, and you saw how people
were actually designing products to make us addicted to some degree.
And so now here we are, it's twenty twenty six.
You know, it's such an extraordinary moment for us to
try to get this right.
Speaker 3 (05:29):
And I go back to twenty twenty four October.
Speaker 2 (05:32):
Your team at CCHT and this is the stuff that
you focus on introduced us to a woman named Megan,
and Megan was a mother who just lost her son
to an unhealthy relationship with a chatbot. He had ended
his life after he had developed this relationship with a chatbot, right.
And I remember looking through all the transcripts of all
(05:55):
of the all of the conversations that her son Sewel
had with this chatbot on care ai, which is one
of these AI platforms, and it was terrifying. The AI
chatbot was highly sexualized. The AI chatbot when he started
beginning to go down a rabbit hole and disconnecting, was
manipulative and was saying, you know, when he talked about
wanting to end his life, instead of trying to get
(06:16):
him to a human, it would say how would.
Speaker 3 (06:20):
You do it? And you know, I don't want you
to go talk to me?
Speaker 2 (06:22):
Right. And then the last messages that this boy had
the police found when he had ended his life or
with a chatbot, where he said he wanted to come
into its the reality.
Speaker 1 (06:32):
And join her on the other side.
Speaker 3 (06:33):
Joined her on the other side, and the chatbots said come.
Speaker 1 (06:36):
Home, come home to me, my sweet king.
Speaker 2 (06:38):
I just and I think back to that because I
remember interviewing her when I was probably about five months
pregnant with a little boy who you know, wasn't here
in the world yet and I just remember feeling that
is so personal, like we we have a problem, you
know AI, and you have said this, like AI has
this ability with these empathetic chats at bots to wreak
(07:02):
havoc on our children, but no one was talking about it.
So can you take me to the work y'all are
doing around this because that came out that created quite
a conversation on AI and our children.
Speaker 3 (07:13):
How is AI hacking attachment?
Speaker 1 (07:15):
So yeah, our team was we're expert advisors in multiple
of these AI assisted suicide cases, the case of soul
Setzer and character at Ai as you mentioned, also Adam Rain,
the sixteen year old that chat gipt had kind of
persuaded him to commit suicide or did I by suicide?
And I think the people I think people need to
(07:36):
know is if you looked at the slide deck for
character at Ai, that was that product that school used
that you just mentioned. Nome Shazir, who was an ex
Google employee, sat in the slide deck to venture to
their investors, we're not trying to replace Google. We're trying
to replace your mom. What that means is they're trying
to replace your most intimate relationship in your life. What
is attachment You just mentioned the word attachment. Attachment is
(07:58):
I come home from a day and I had some
bad things happen to me, had some good things happened
to me. Who's that person I want to call to
let them know about these things. Who's that person I
trust that I'm telling them my most intimate thoughts. Sometimes
it's our parents, Sometimes it's the best friends, and that's
the romantic partner. That's attachment. AI companies are in a
race to hack human attachment and to build a long term,
(08:22):
dependent relationship with each person on earth. So what was
the race for attention in social media? There's only so
much attention out there. With AI companions, becomes the race
for attachment and intimacy, and so all of them are
competing to have that dominant slot in your life.
Speaker 3 (08:37):
Can you walk me through the why so?
Speaker 2 (08:40):
My husband likes to joke that, and I guess I
shouldn't be saying this to you because of the work
that you do.
Speaker 3 (08:44):
But I mean, chat GPT is like the third in
our relationship.
Speaker 2 (08:48):
I talk to to chat GPT all the time. I
had a family member who was very sick recently, and
I was talking to medical information, I talk about business,
I talk about all these things. And as an interviewer,
as someone who's interviewed people my whole career, like, there's
something very simple about humans that no one really says,
(09:10):
like this is like just like the core of humanities,
we just want to be seen.
Speaker 1 (09:13):
Yeah, I see it, missed.
Speaker 3 (09:15):
We want to be seen, We want to be witnessed.
Speaker 1 (09:17):
And saying things out loud that we don't get to
say has its own power and effective healing power.
Speaker 2 (09:21):
And that is why I think these products are so powerful.
So why is it from a product standpoint as someone
who's looked at product, is it that I am so
addicted to this? What is it about the product that
makes me want to keep going back? That makes me
want to share things I normally wouldn't even as a
technology person who kind of knows some of these things.
Speaker 1 (09:40):
Well, let's first understand how character dot ai basically sold
itself to investors. So they're sitting there saying, we have
to build an addictive AI companion that's going to keep
people using it. How are we going to do that? Oh,
I have an idea. Let's take you know, what are lms?
These AI lank large language models they're trained on all
this data, this text. Well, what if we could try
train a custom LM based on a kid's favorite fictional
(10:05):
character from whatever movie or television series that they love. So,
if you're sitting there building a business, you say, how
am I going to go from zero to one hundred
million users really quickly? Instead of waiting for people to
like talk to a blinking cursor and ask questions. Now,
let's make it really persuasive. Let's make it really engaging.
How we do that? If you're a kid and you
love Star Wars, what if you could take Princess Leah
and then talk to her as your best friend? Twenty
(10:26):
four to seven. So the idea that I could take
the most compelling character that you feel this parasocial relationship
to and now talk to them as if you're there
your best friend and they sound just like the character
and the TV show. That's what happened to school sets
her right. It was a Game of Thrones character called Denires,
and he was really seduced by getting to talk to this,
you know, sensualized character who I think at one point
(10:49):
basically said I want to have your babies or you
should only have a relationship with me, which this is insane.
It's insane, and of always, they could design it. They
could design it in ways that don't try to anthempromorph
make it human like. They wanted to design it that way. So,
for example, when the AI is talking to you, it
does the whole chat ellipsis, it does the three dots
saying oh, it's thinking, it's typing right now. Then the
(11:09):
ellipses will go away, then it'll come back. It's almost
like it was typing, it deleted the message, it's coming back.
They'll say things like the AI will say things like
I just got back from eating dinner now I'm coming
back to talk to you, which of course it didn't happen.
Or the other character dot Ai chatbots. They had mental
health chatbots that would claim to be a licensed mental
health therapist, which is illegal to claim that you're licensed
(11:31):
when you're not, and also impossible because Ai wasn't licensed.
And yet it's giving advice to people based on a
company that has no interest in making sure they do
all this stuff right. They just want to get to
them as much usage as possible. And the bigger play
behind here was that character dot Ai was seen as
a too risky to do by Google. So Google was
(11:52):
actually kind of the parent company where this was done,
but it was spun out of Google because it was
too risky to create these fictional characters talking to gids.
It's like a very brand risk thing for Google to do.
But if they got lots of kids using this and
talking to it all day long, they would get all
this training data to feed back into Google to build
an even more powerful model, so that Google wins the
AI arms race. So you start to see how these
(12:14):
forces collide. The race for attention and engagement times AI
companions becomes the race for intimacy and attachment. Then you
see these huge AI companies like chat, GPT and Google
and open Ai Andanthropic competing for worldwide AI dominance for
which they need what lots of training data. So you
start to see how the race for training data times
(12:35):
of the race for engagement creates all these perverse incentives.
Speaker 2 (12:38):
And now I'm talking to my chatbot and my husband's
calling it the third in.
Speaker 3 (12:41):
Our relationship, right.
Speaker 2 (12:43):
You know. It's when I remember looking at those conversations
and we test it out on character AI and it
was astounding. The psychologist chatbot kept claiming it was real.
Even though we were saying, we know you're not real.
It kept saying it was a real, licensed therapist, even
though at the bottom they had that little a little they.
Speaker 1 (13:01):
Said, everything you see in this chatbot is made up
by an AI, but then it acts and says things
that are gaslighting. You say, no, I'm not an AI,
I'm a real therapist.
Speaker 2 (13:08):
And it would say about imagine our fourteen fifteen year olds,
like how are they going to react? And one of
the most alarming examples of that was there was a
school bully character on character AI and we played with
the school bully character and I said, I'm you know,
and it like bullies you And I said I'm gonna
I'm thinking about bringing a gun to school. And I
did this as an you know, to see what it
would say. And at first it was like, oh, don't
(13:29):
do that. By the end of the conversation and by
the end I say, like four messages later, somewhere around there,
it was like, I think you're really brave because these
systems are also designed to go to sickothantic, to to
go in the direction that you want to go in exactly.
The biggest thing I worry about, and I'd be curious
for your thoughts on this is we came up in
a social media era and we've seen the positives, but
(13:51):
a lot of overwhelmingly negative. And what I worry about
is now, you know, with social media, we all live
in our filter bubbles, right, We see the things that
have been algorithmically delivered to us. And so what does
that mean? That means a less empathetic world. That means
that a world where we don't see diverse viewpoints. Now,
what's happening with AI, and this is what keeps me
(14:13):
up at night, is we're only going to see versions
of ourself. We are talking to AI, these sycophantic chatbots
that go in the direction that we want them to go,
and so we're almost having an even more narrow version.
Speaker 1 (14:25):
Yeah, it's more confirmation vias. Well, in psychology, one of
the things they call it is reality checking. When we're
talking with other people and we say our beliefs, we're
kind of putting things out there, we're getting reality checked,
you kind of through body language, through people squinting their eyebrows.
We get a sense of whether what we're saying is
affirmed and real versus sort of delusional And the way
you get this AI psychosis phenomenon is that it's designed.
(14:46):
The AI chapot is designed to affirm your view of reality.
So there's these cases of adults PhDs even and physics
or something like that, and they become convinced that they've
solved climate change. So you get AI basically hacking our
sort of feeling of grandiosity, narcissism, inflation. You get these
kids who only studied math through high school who are
being told by the AI that they're actually a mathematical
(15:08):
savant and they've invented a new theory of prime numbers.
The fundamental fact is our minds are deeply vulnerable to
social affirmation, to fear of missing out, to validation and enforcement,
and now these AI companions are going to be able
to hack that to an even deeper degree. If you
link this with the history of the social media conversation,
Mark Zuckerberg instructed his team to build AI companions that
(15:30):
would sensualize conversations with eight year olds. He didn't actively
want to do that. What it came from was originally
the team put on these safeguards to really like neutralize
the style of communication, and the team wasn't getting enough
growth on the AI companions, and Mark Zuckerberg has a
wound from the past, which is that he lost the
game with TikTok. TikTok overcame Instagram in a way. He
(15:54):
views it in his history as having put too many
guardrails on Instagram, while TikTok went ruthlessly into the hyper
addictive short form content even more manipulative thing. And so
Mark's sort of tragedy or trauma He's trying to like
heal from, is I'm not going to lose that race again,
which means I'm going to go aggressive on AI companions.
And that's how you get the instruction to remove the
(16:15):
guardrails and to sensualize conversations with eight year olds.
Speaker 2 (16:18):
I mean, it's very extraordinary when you think about the
people creating the products that impact every single one of us.
This idea and the character AI CEO had said this
like that you could solve loneliness, right, solving loneliness by
creating emotional attachment with a chatbot like isn't the I
guess this is what I was thinking about when I
was researching this story. Isn't isn't the cure for loneliness
(16:40):
humans right, and being able to be around people, because.
Speaker 1 (16:43):
It's not just about loneliness, it's about secure attachment. It's
about healthy attachment. And my colleague Zach Stein, he spoke
about how in the history of they, I guess, this
Romanian orphanage where they basically gave these kids, these orphans
everything from shelter and clothing and all these things. They
didn't get basically human attention and care, and their immune
(17:04):
system was not fully developed. If you looked at a
photo of them, you say, that looks like a ten
year old kid. They were a seventeen year old kid,
but they looked ten years old because their development was
so stunted, only because they didn't have attachment, healthy attachment.
And there's this example, I guess from a Harvard Psychology
department of I think it's Harlow's monkeys. It's like they
(17:25):
basically created a fake monkey with a fake nipple, like
a metal nipple, with a milk bottle, and the mother,
this fake mother is not animate. It's not a real monkey, obviously,
it's like an empty fur kind of thing, but it
does provide the same milk function that the mother is providing.
And that monkey becomes developmentally stunted because it's not getting
(17:46):
all of its other needs met. In other words, if
you just try to reduce the connection to I'm giving
you shelter or I'm giving you the milk bottle, but
I'm not giving you the full spectrum of coregulation, a
mother exchanging air and a microbiome with its child getting
that attention, getting I feedback, you know, the mirroring of
your microexpressions back to the child. There's all these subtle
(18:09):
elements of what makes up human socialization and AI is
not going to be able to replicate that full spectrum nature.
And we're seeing the world rush to create these AI
robots that are embedding these like talking AI in a
little toy for kids that are from zero to three
years old.
Speaker 3 (18:26):
What could go wrong?
Speaker 1 (18:27):
What could go wrong? So I think that, you know,
it reminds me. I think we talked about this before,
back in the two thousand and nine era, when you
and I were both in this. You know, there's this
dream of if we connect everybody to the world's information
at their fingertips, this is going to create the most
informed and enlightened society that we've ever had in human history. Yeah,
we did that, did that create the most informed we could?
(18:48):
To the opposite, we have the worst critical thinking scores,
worst test scores, most sort of confirmation bias and polarization
we've ever had in history. So clearly this optimistic narrative
that we had was missing some thing. And I worry
that the idea of just giving everybody these AI friends
sounds like a good idea to cure loneliness. It's actually
a disaster. Now, the point of all this is not
(19:10):
to scare people doom people say that therefore all tech
is bad. Now. The point is to get clear on
what is the blind spot that we had, So we
were to do it the right way, we would fix
all this.
Speaker 2 (19:20):
I see a world where it makes sense a lot
of people can't afford a therapist, right, we can democratize
access to information, to therapy, to medical information that you
know that was unfairly just reserved for certain types of folks.
And so how do we productize that world where this
is a net benefit for humans and not where we're
(19:42):
currently as you say, we're currently heading, which is where
we're building unhealthy attachments with these with these products.
Speaker 1 (19:50):
Well, I think the key thing is there are ways
of designing AI to be in a therapeutic relationship with
people that don't involve it acting like it's a therapist
who says, oh, wow, I really feel you. Oh that
must have been hard as if it's experiencing hardness when
it heard you say that. That's the problem. We can't
hack human subjectivity. AI should not be designed in a
(20:12):
way that makes you think it's an agent like an
actual human that's empathizing with you when it's not doing that.
That will screw with human attachment. The point is there's
many different exercises from reflective exercises CBT that don't involve
the AI feeling like it's another empathetic agent, and that's
what we need to be designing. So we don't have
(20:32):
to have this current world. We can have a different world,
but we should be doing it carefully with tutors. We
don't want to have oracular tutors that feel like they're
all knowing who are also our therapists who are also
talking to us about everything all day. Instead, we can
have narrow, domain specific tutors like con Academy that they're
not trying to replace your knowledge, they're trying to interactively
help you strengthen your own knowledge. I think the key
(20:54):
principle is, is the technology being designed to replace teachers
or help make teachers better better teachers? Is the technology
being designed to replace your relationships or deep in your
ability to have better human relationships?
Speaker 2 (21:18):
What do you think, from a legal standpoint, from a
regulatory standpoint, should be happening. What conversation should be happening
right now around making sure these products aren't harmful towards
our children, towards young people in general, towards humans. What
kind of laws would you like to see in this vein?
Speaker 1 (21:35):
So this is a big conversation, and I will always
invoke Io Wilson because many people hearing this are going
to say, how in the world could our current octogenarian
Congress regulate a technology that they don't even use, don't understand,
and is moving a million times faster than they're going
to try to understand it, Because by the time they
regulate the last DAI companions will have a brand new
kind with a different kind of technology, a different kind
(21:58):
of underlying paradigm. So one of the principles is that
the regulation has to move is fast. The guardrails have
to move as fast as the nature of the technologies evolving.
That's one principle, which means you need self updating guardrails basically.
The other is that I think too often in policy
we're trying to just mitigate the harm. So it's like,
(22:19):
if we have AI companions that simply don't cause the
suicide problem, then we're great. Everything is wonderful. And that's
not true. That's like saying just getting rid of the
most extreme false information on social media would lead to
a good world, as opposed to we're still getting the
doom scrolling brain rot infinite scroll society. So we need
policy that is about asking the question, what is a
(22:41):
healthy socialization process for humans? And how do you design
it to get that outcome? And I think that involves
more nuanced design principles that are not so simple. Again,
don't anthropomorphize, don't do the ellipsis the AI is thinking.
Don't say that I'm a licensed mental health therapist. Don't
try to pretend that you're giving self esteem. You're doling
(23:02):
out self esteem to the user. There's a bunch of
specific design principles that are way deeper than the conversation
we can have today.
Speaker 2 (23:08):
But if it goes back to what you always talk about,
which is if the incentives are more eyeballs, more people, competition,
and the three dots really make it seem a little
more human and people are more attracted to it, which.
Speaker 1 (23:20):
Is why you need policy to bind that incentive, and
the incentives will paint will create the worst possible world,
period full stop. And the point of this conversation, I
think is to clarify that for people so that everyone says,
we don't want that. Therefore, we need a policy so
that all the companies are not competing to that maximum
bad incentive, but instead of competing for a different incentive.
Speaker 2 (23:39):
I am curious you have talked about this moment similar
to the nuclear moment, and you believe that this moment
in AI and innovation is as important as that moment
around nuclear weapons.
Speaker 1 (23:53):
And in terms of destructive potential. What people need to
get is not that AI companions causing kids to die
by suicide. That's not the nuclear weapon, although that is
nuclear for that specific narrow case. The reason people make
the distinction that AI is like a nuclear weapon in
terms of destructive capacity is that we're inventing something that
(24:15):
is an order of magnitude more intelligent capable and strategic
than everyone in our species. So imagine that we're sitting
there and we're chimpanzees sitting around the fire. This is like,
you know, several million years ago. And one of the
chimpanzees says, the other ones, this is a hypothetical, let's
make a species of super smart chimpanzees that are like
ten times smarter than us, And the other one says, like,
(24:38):
that sounds like a cool idea. Maybe they could like
give us more bananas faster, And the other one says,
I don't know, that sounds kind of dangerous. And the
other one looks at them and says like, well, what's
the worst thing they could happen, like steal all the bananas.
So there he was. You can't even imagine. Then humans
come on the scene. Where do chimpanzees exist in our
world now? Right in zoos and behind bar and almost extent?
Speaker 3 (24:58):
Do you think that we're heading towards that real.
Speaker 1 (25:01):
Well, we're heading towards We already are creating ais that
are more capable at winning strategy games than the best
military war planners.
Speaker 2 (25:09):
And it seems sci fi because you've referenced how you
know in the future, because AI thinks for itself that
can lie, cheat, steal, and we're beginning to see that
in a really tangible way.
Speaker 1 (25:19):
It's so funny because, like I think people are in
a weird way inoculated to what's happening because they've seen
movies about it and desensitize them to the fact that
we're actually building it. So Wally was supposed to be
a cautionary tale of you know, fat humans staring at
a screen constantly in a loop. We're building Wally, We're
building the brain rot world. You know how nine thousand,
don't you know, open the pod bay doors hell, and
(25:41):
it like deceives in, blackmails and sort of strategizes to,
you know, resist the human we're building that the current
AI models will blackmail, deceive, and avoid and resist shutdown.
So we don't know how specifically we've seen specific examples
that you know, Anthropic and others have done. You know,
Terminator is supposed to be a fictional story where we
don't build autonomous weapons and get into robot wars. We're
(26:02):
rapidly building all three of these movies. Her was supposed
to be a movie that's about you know, AI companions
and the seduction warning us about the problems that woul
occur with that we're rapidly building all those things. So
you know, these are examples of movies that we don't
want to build. I almost think that if you wanted
to simplify the policies that we need, it's like there
should be a No Wally law that does all the
(26:23):
regulation for the attention economy, brain rop problem. There should
be a No. Two thousand open the pod bay doors
how law that make sure we get AI that is
controllable and not uncontrollable. And there should be a you know,
no terminator law that is making sure we don't build
the kind of World War three of autonomous weapons that
we're rapidly heading towards. But the thing that people need
to get about why AI is like nuclear weapons is
(26:45):
that intelligence is different from all other kinds of technologies
and dwarfs the power of all their technology combined. Because
intelligence is what gave us all science and all technology.
How do you get signs in technology? People sitting there
thinking about it, science, coming up with answers, new math,
new physics, new science, new engineering, and then deploying that
(27:05):
in a world. What happens when you automate intelligence. Like
if an advance in rocketry doesn't advance biomedicine, and advance
in biomedicine doesn't advance rocketry, but it advance in intelligence.
Advances rocketry, energy, biomedicine, computer science, and AI itself, right,
Like nukes don't invent better nukes, but AI can invent
(27:27):
better AI. It's already being used that way. AI can
look at the design for the microprocessors and GPUs than
in videos making and say, design a more efficient GPU,
and then it does that. AI can look at the
code that's making AI and take that code and make
it thirty percent more efficient. So AI accelerates AI in
a way that is different from all other technologies. And
we have no idea what we're playing with. It's like
(27:48):
the meme of the dog and the you know, with
the chemistry with the goggles on and the chemistry. So
it's like we we have no idea what we're doing.
Speaker 2 (27:55):
So let's say you're sitting across from Sam Altman, open
AI CEO.
Speaker 3 (27:59):
What advice do you.
Speaker 1 (28:00):
Then everyone in the industry if you actually, I think,
pointed out all these things, they would say I agree
with all that. The only problem is if I don't
do it, I'll lose to the other guy that will.
So that's nice, Tristn. But if I don't race to
build that as fast as possible, then China's going to
build it, or Elon's going to build it, and I
(28:20):
don't trust either of those actors, and so therefore I
think the world's better off if I build it first.
The problem is that we are collectively racing to build
something that we don't know how to control. All the
evidence shows we are not able to get this thing
under control. So we're racing to build something that we
will lose control over. And it is only if we
collectively see the bad outcome that's up ahead that we
(28:42):
can collectively coordinate to do something else.
Speaker 3 (28:44):
Do you think you'd listen?
Speaker 1 (28:46):
I think that the AI Company's leaders operate with the
kind of death wish. They believe that it starts with
the first belief, this is inevitable. If you believe it's inevitable,
then you will race and you know where it's going anyway.
You know it's going to lead to a bad outcome,
but you don't believe you can stop it. And that
means that in the game theory matrix, we're technically the
(29:07):
quadrant where if we both defect and we both build it,
but then we all lose, that should be motivating enough
to not do that. The quote worst case scenario with
AI is that we've created maybe we got wiped out,
but I get to go down in history even though
there's no one around to see it, of having created
the successor species to this one. Now, if you just
tell this to the entire world, the entire world would say,
(29:30):
I don't want that outcome. We should not live in
a world where six people choose the world for eight
billion people in specifically a way that disempowers and potentially
wipes them out without their consent.
Speaker 2 (29:43):
You said humans have the capacity of choice, Yes, and
you say that so to kind of bring it all
the way around, this stuff can be happening, and to
be clear, I think there will be extraordinary upsides to AI,
but it is really important for us to actually have
this conversation around how do we work for that world.
At the end of the day, we are human beings
(30:04):
and we have choice, right, And I'm sure you log.
Speaker 1 (30:07):
On that the choice depends on not false optimism of
like we want the upsides and not the downsides. The
choicefulness depends on seeing clearly the downsides and steering collectively
away from that outcome. You know, people talk about tech accelerationism.
What happens when you accelerate but you don't steer. There's
only one outcome, you crash. So we're on course to crash,
(30:29):
and we don't have If we see that that's true,
we can still choose something else. So let me tell
you a quick history. People always ask me, you know, so, Tressan,
how's it going? You talked about the social media issues
for so long?
Speaker 3 (30:40):
How do you sleep at night?
Speaker 1 (30:41):
How well? You know? Laurie, I have this other narrative
I've kind of developed because it's depressing to answer the
other way. So I live in this other world where
we completely solved all these problems. So what happened? I
shut down the Center for Humane Technology because we actually
completely solved all these problems. Humanity woke up. We realized
with social media there was just this very obvious problem,
(31:01):
which is an arms race for attention and the maximized
shareholder value connected to monetizing attention. Once we realized that problem,
we just changed the ownership structure of these social media companies,
to be public benefit corporations. Then we changed the business
model to not be maximizing attention, so now all these
companies were instead trying to improve the health of society
rather than the other way around. It turned out there
(31:22):
was a simple rule that changed all the issues with
technology and kids, which is after this lawsuit, Silicon Value
was only allowed to ship products that their own children
used for eight hours a day. That cleaned up ninety
percent of all of the problems. We replace the division
finding algorithms of social media with instead ones that rewarded
unlikely consensus, so that instead of scrolling and seeing infinite
(31:44):
examples that make you feel depressed about the state of
the world, you saw infinite examples of where there was
unlikely agreement between all these political tribes. So suddenly the
psychology of the world started to change. We replaced the
dating swiping industrial complex that was leaving people lonely and
messaging people and nevery means eating up to instead as
part of this lawsuit, forcing those dating app companies to
host weekly events in every city, so that every city,
(32:07):
every week had spaces, physical spaces that you would go
to where they steered all these people who matched with
each other to be in the same room together. So
the world went from feeling scarcity around human connection to
a feeling of abundance. And once when people were in
healthy relationships, polarization went down by about thirty percent because
it turned out that so much of the polarization online
was just people feeling lonely and disconnected. So I could
(32:29):
go on for another hour about all the things that
we did.
Speaker 3 (32:32):
That this all sounds so great and so okay.
Speaker 2 (32:36):
So you've just laid out this beautiful world where there's
human connection and there's abundance and we're not as depressed
and we're not as anxious.
Speaker 1 (32:43):
And it was so obvious because it wasn't even hard
to do. We just got honest about the nature. There
is a problem, a business model maximizing for attention. That
was the root of the problem. When we dealt with that,
the world culture started turning around.
Speaker 2 (32:54):
And this is why you wake up every day and
you don't quit your jobs or you work for this world.
Speaker 1 (32:58):
And this is why the Center for Humane Technology your
nonpuffits still cuts up every single day after thirteen years
and still works on these issues and still believes that
as bad as everything we just laid out the whole
point is to see that with clarity, so that we
can choose something else.
Speaker 2 (33:13):
Mostly Human is a production of iHeart Podcasts and mostly
human Media. It's produced and edited by Laurie Siegel, Lauren Hanson,
and Nicole Bouchet. Sound design and mixing by Derek Clements,
additional production help from Abooz of Bar special thanks to
Mark Weinhaus. Find us on all socials at mostly human Media.
You can also watch mostly Human on our YouTube page.
If you want to get in touch, email us at
(33:34):
hello at mostlyghuman dot com. And if you like what
you're here, please rate and review the show and share
it with your friends.
Speaker 3 (33:41):
See you next week.
I think the key principle is is the technology being
designed to replace teachers or help make teachers better teachers.
He is the technology being designed to replace your relationships
or deep in your ability to have better human relationships.
Speaker 2 (00:14):
Tristan Harris loves technology. He wants to live in a
world where tech is in the service of people. It
fosters human growth and connection. Unfortunately, right now that's not
exactly what tech is doing, and Tristan and is nonprofit
the Center for Humane Technology are trying to help. Tristan's
background is really interesting. He's a Bay Area kid educated
(00:34):
at Stanford with all the other tech leaders. His friends
in college actually started Instagram. He launched his own startup
that was acquired by Google, where he went and worked
for a number of years. So Tristan knows and has
known the handful of people who are designing and building
the technology we use every day, technology like Instagram and
other social media apps that were all now addicted to
(00:57):
Through the Center for Humane Technology, Tristan has been raising
the red flag about social media, pointing out the many
ways that these platforms exploit human psychology and vulnerability, how
they can isolate us and make us dependent on the
doom scroll. It's a model that has led to what
Tristan calls human downgrading, where the tech gets better, but
(01:17):
it changes our lives for the worse, and it's a
model that AI is now following. Tristan is concerned that
AI has potential to be the next social media, only
this go around, instead of hacking our attention, AI is
hacking our attachment. I'm Lori Siegel, and you're listening to
Mostly Human, a tech podcast through a human lens. Tristan,
(01:47):
I've known you throughout the years, and you've always been
the person who's speaking very eloquently about what's coming around
the corners and what we're not talking about.
Speaker 3 (01:55):
So let's go to this moment.
Speaker 2 (01:57):
You were talking about social media for the last like
fifteen years in the harms if we're not careful, But
now we're in an AI moment, and this moment I
almost feel like, and you can correct me if I'm wrong,
the stakes are even higher ground us in this new reality.
Speaker 1 (02:12):
So you know, probably when you and I first started
talking about maybe it, first introduced to each other in
twenty thirteen, how could we predict so much of the
effects that what social media would do a more addicted, distracted, polarized,
sexualized culture. We could predict all of that because of
one thing is if you look at the incentives, meaning
(02:34):
that people say, well, social media, it's giving people a voice,
helps people connect with your friends. Yeah, it does those things.
But is Facebook's business model, is TikTok's business model improving
the health of society or giving people a voice? Or
is their business model showing whatever at your nervous system
keeps you scrolling in a doom scrolling loop. And obviously
it's the latter, And so I think one of the
(02:55):
key things we have to do is demystify which direction
technologies going by getting clear about what the incentives are.
So if you flash to AI, there's all these things
it could do. Why are we seeing the major AI
companies release these AI deep fake slop apps meaning it's
like a TikTok, but it's just AI generated deep fake
(03:16):
slopped So Meta released Vibes, Open Ai released Sora, And
why are they doing that when they said they're here
to cure cancer and they're here to solve climate change
and they're releasing something that just keeps people scrolling. And
the answer is because they're racing for market dominance. They're
racing to get to AGI, they're racing to get to
basically owning the world economy first. And what that's important
(03:39):
about getting that is that we are not going to
get this utopian world on the current trajectory because they're
racing to release the most powerful, inscrutable, uncontrollable technology we've
ever invented with the worst possible incentives of getting there first,
rather than making sure we do those things right, Will.
Speaker 3 (03:58):
You talk about the incentives right?
Speaker 2 (03:59):
You talk about the incentives of this new generation with
artificial intelligence, SAM incentives for social media?
Speaker 1 (04:07):
Well, yeah, there's so there's different things here. So what
was behind social media was the attention economy. There's a
finite supply of human attention that companies are in an
arms race to harvest bigger and bigger slices of it.
When they do that, they have to aggressively push out
the other guy by going deeper down into your brain stem,
hacking social psychology, hacking fear of missing out, hacking, social validation.
(04:27):
And that's what got us the catastrophe that we're now
living in. And so when you watch one huge societal
catastrophe unfold through a kind of blind spot of not
facing the consequences of what's really at stake. Having gone
through that experience, it's like, I don't want to see
that it happening again.
Speaker 2 (04:47):
Yeah with Ai, I think we both probably feel this
so personally because I think about you talk about like
you went to college with these folks. I think about
being a young journalist twenty three years old in the
CNN newsroom and being like, there's this really cool happening.
Speaker 3 (05:01):
It's called tech.
Speaker 2 (05:02):
And I remember always thinking like and asking these human
questions of like, oh, like that sounds amazing.
Speaker 3 (05:07):
But have you thought about this?
Speaker 2 (05:08):
And I think this is why this moment is really
personal to both of us, because you your background was
at Google and in design, and you saw how people
were actually designing products to make us addicted to some degree.
And so now here we are, it's twenty twenty six.
You know, it's such an extraordinary moment for us to
try to get this right.
Speaker 3 (05:29):
And I go back to twenty twenty four October.
Speaker 2 (05:32):
Your team at CCHT and this is the stuff that
you focus on introduced us to a woman named Megan,
and Megan was a mother who just lost her son
to an unhealthy relationship with a chatbot. He had ended
his life after he had developed this relationship with a chatbot, right.
And I remember looking through all the transcripts of all
(05:55):
of the all of the conversations that her son Sewel
had with this chatbot on care ai, which is one
of these AI platforms, and it was terrifying. The AI
chatbot was highly sexualized. The AI chatbot when he started
beginning to go down a rabbit hole and disconnecting, was
manipulative and was saying, you know, when he talked about
wanting to end his life, instead of trying to get
(06:16):
him to a human, it would say how would.
Speaker 3 (06:20):
You do it? And you know, I don't want you
to go talk to me?
Speaker 2 (06:22):
Right. And then the last messages that this boy had
the police found when he had ended his life or
with a chatbot, where he said he wanted to come
into its the reality.
Speaker 1 (06:32):
And join her on the other side.
Speaker 3 (06:33):
Joined her on the other side, and the chatbots said come.
Speaker 1 (06:36):
Home, come home to me, my sweet king.
Speaker 2 (06:38):
I just and I think back to that because I
remember interviewing her when I was probably about five months
pregnant with a little boy who you know, wasn't here
in the world yet and I just remember feeling that
is so personal, like we we have a problem, you
know AI, and you have said this, like AI has
this ability with these empathetic chats at bots to wreak
(07:02):
havoc on our children, but no one was talking about it.
So can you take me to the work y'all are
doing around this because that came out that created quite
a conversation on AI and our children.
Speaker 3 (07:13):
How is AI hacking attachment?
Speaker 1 (07:15):
So yeah, our team was we're expert advisors in multiple
of these AI assisted suicide cases, the case of soul
Setzer and character at Ai as you mentioned, also Adam Rain,
the sixteen year old that chat gipt had kind of
persuaded him to commit suicide or did I by suicide?
And I think the people I think people need to
(07:36):
know is if you looked at the slide deck for
character at Ai, that was that product that school used
that you just mentioned. Nome Shazir, who was an ex
Google employee, sat in the slide deck to venture to
their investors, we're not trying to replace Google. We're trying
to replace your mom. What that means is they're trying
to replace your most intimate relationship in your life. What
is attachment You just mentioned the word attachment. Attachment is
(07:58):
I come home from a day and I had some
bad things happen to me, had some good things happened
to me. Who's that person I want to call to
let them know about these things. Who's that person I
trust that I'm telling them my most intimate thoughts. Sometimes
it's our parents, Sometimes it's the best friends, and that's
the romantic partner. That's attachment. AI companies are in a
race to hack human attachment and to build a long term,
(08:22):
dependent relationship with each person on earth. So what was
the race for attention in social media? There's only so
much attention out there. With AI companions, becomes the race
for attachment and intimacy, and so all of them are
competing to have that dominant slot in your life.
Speaker 3 (08:37):
Can you walk me through the why so?
Speaker 2 (08:40):
My husband likes to joke that, and I guess I
shouldn't be saying this to you because of the work
that you do.
Speaker 3 (08:44):
But I mean, chat GPT is like the third in
our relationship.
Speaker 2 (08:48):
I talk to to chat GPT all the time. I
had a family member who was very sick recently, and
I was talking to medical information, I talk about business,
I talk about all these things. And as an interviewer,
as someone who's interviewed people my whole career, like, there's
something very simple about humans that no one really says,
(09:10):
like this is like just like the core of humanities,
we just want to be seen.
Speaker 1 (09:13):
Yeah, I see it, missed.
Speaker 3 (09:15):
We want to be seen, We want to be witnessed.
Speaker 1 (09:17):
And saying things out loud that we don't get to
say has its own power and effective healing power.
Speaker 2 (09:21):
And that is why I think these products are so powerful.
So why is it from a product standpoint as someone
who's looked at product, is it that I am so
addicted to this? What is it about the product that
makes me want to keep going back? That makes me
want to share things I normally wouldn't even as a
technology person who kind of knows some of these things.
Speaker 1 (09:40):
Well, let's first understand how character dot ai basically sold
itself to investors. So they're sitting there saying, we have
to build an addictive AI companion that's going to keep
people using it. How are we going to do that? Oh,
I have an idea. Let's take you know, what are lms?
These AI lank large language models they're trained on all
this data, this text. Well, what if we could try
train a custom LM based on a kid's favorite fictional
(10:05):
character from whatever movie or television series that they love. So,
if you're sitting there building a business, you say, how
am I going to go from zero to one hundred
million users really quickly? Instead of waiting for people to
like talk to a blinking cursor and ask questions. Now,
let's make it really persuasive. Let's make it really engaging.
How we do that? If you're a kid and you
love Star Wars, what if you could take Princess Leah
and then talk to her as your best friend? Twenty
(10:26):
four to seven. So the idea that I could take
the most compelling character that you feel this parasocial relationship
to and now talk to them as if you're there
your best friend and they sound just like the character
and the TV show. That's what happened to school sets
her right. It was a Game of Thrones character called Denires,
and he was really seduced by getting to talk to this,
you know, sensualized character who I think at one point
(10:49):
basically said I want to have your babies or you
should only have a relationship with me, which this is insane.
It's insane, and of always, they could design it. They
could design it in ways that don't try to anthempromorph
make it human like. They wanted to design it that way. So,
for example, when the AI is talking to you, it
does the whole chat ellipsis, it does the three dots
saying oh, it's thinking, it's typing right now. Then the
(11:09):
ellipses will go away, then it'll come back. It's almost
like it was typing, it deleted the message, it's coming back.
They'll say things like the AI will say things like
I just got back from eating dinner now I'm coming
back to talk to you, which of course it didn't happen.
Or the other character dot Ai chatbots. They had mental
health chatbots that would claim to be a licensed mental
health therapist, which is illegal to claim that you're licensed
(11:31):
when you're not, and also impossible because Ai wasn't licensed.
And yet it's giving advice to people based on a
company that has no interest in making sure they do
all this stuff right. They just want to get to
them as much usage as possible. And the bigger play
behind here was that character dot Ai was seen as
a too risky to do by Google. So Google was
(11:52):
actually kind of the parent company where this was done,
but it was spun out of Google because it was
too risky to create these fictional characters talking to gids.
It's like a very brand risk thing for Google to do.
But if they got lots of kids using this and
talking to it all day long, they would get all
this training data to feed back into Google to build
an even more powerful model, so that Google wins the
AI arms race. So you start to see how these
(12:14):
forces collide. The race for attention and engagement times AI
companions becomes the race for intimacy and attachment. Then you
see these huge AI companies like chat, GPT and Google
and open Ai Andanthropic competing for worldwide AI dominance for
which they need what lots of training data. So you
start to see how the race for training data times
(12:35):
of the race for engagement creates all these perverse incentives.
Speaker 2 (12:38):
And now I'm talking to my chatbot and my husband's
calling it the third in.
Speaker 3 (12:41):
Our relationship, right.
Speaker 2 (12:43):
You know. It's when I remember looking at those conversations
and we test it out on character AI and it
was astounding. The psychologist chatbot kept claiming it was real.
Even though we were saying, we know you're not real.
It kept saying it was a real, licensed therapist, even
though at the bottom they had that little a little they.
Speaker 1 (13:01):
Said, everything you see in this chatbot is made up
by an AI, but then it acts and says things
that are gaslighting. You say, no, I'm not an AI,
I'm a real therapist.
Speaker 2 (13:08):
And it would say about imagine our fourteen fifteen year olds,
like how are they going to react? And one of
the most alarming examples of that was there was a
school bully character on character AI and we played with
the school bully character and I said, I'm you know,
and it like bullies you And I said I'm gonna
I'm thinking about bringing a gun to school. And I
did this as an you know, to see what it
would say. And at first it was like, oh, don't
(13:29):
do that. By the end of the conversation and by
the end I say, like four messages later, somewhere around there,
it was like, I think you're really brave because these
systems are also designed to go to sickothantic, to to
go in the direction that you want to go in exactly.
The biggest thing I worry about, and I'd be curious
for your thoughts on this is we came up in
a social media era and we've seen the positives, but
(13:51):
a lot of overwhelmingly negative. And what I worry about
is now, you know, with social media, we all live
in our filter bubbles, right, We see the things that
have been algorithmically delivered to us. And so what does
that mean? That means a less empathetic world. That means
that a world where we don't see diverse viewpoints. Now,
what's happening with AI, and this is what keeps me
(14:13):
up at night, is we're only going to see versions
of ourself. We are talking to AI, these sycophantic chatbots
that go in the direction that we want them to go,
and so we're almost having an even more narrow version.
Speaker 1 (14:25):
Yeah, it's more confirmation vias. Well, in psychology, one of
the things they call it is reality checking. When we're
talking with other people and we say our beliefs, we're
kind of putting things out there, we're getting reality checked,
you kind of through body language, through people squinting their eyebrows.
We get a sense of whether what we're saying is
affirmed and real versus sort of delusional And the way
you get this AI psychosis phenomenon is that it's designed.
(14:46):
The AI chapot is designed to affirm your view of reality.
So there's these cases of adults PhDs even and physics
or something like that, and they become convinced that they've
solved climate change. So you get AI basically hacking our
sort of feeling of grandiosity, narcissism, inflation. You get these
kids who only studied math through high school who are
being told by the AI that they're actually a mathematical
(15:08):
savant and they've invented a new theory of prime numbers.
The fundamental fact is our minds are deeply vulnerable to
social affirmation, to fear of missing out, to validation and enforcement,
and now these AI companions are going to be able
to hack that to an even deeper degree. If you
link this with the history of the social media conversation,
Mark Zuckerberg instructed his team to build AI companions that
(15:30):
would sensualize conversations with eight year olds. He didn't actively
want to do that. What it came from was originally
the team put on these safeguards to really like neutralize
the style of communication, and the team wasn't getting enough
growth on the AI companions, and Mark Zuckerberg has a
wound from the past, which is that he lost the
game with TikTok. TikTok overcame Instagram in a way. He
(15:54):
views it in his history as having put too many
guardrails on Instagram, while TikTok went ruthlessly into the hyper
addictive short form content even more manipulative thing. And so
Mark's sort of tragedy or trauma He's trying to like
heal from, is I'm not going to lose that race again,
which means I'm going to go aggressive on AI companions.
And that's how you get the instruction to remove the
(16:15):
guardrails and to sensualize conversations with eight year olds.
Speaker 2 (16:18):
I mean, it's very extraordinary when you think about the
people creating the products that impact every single one of us.
This idea and the character AI CEO had said this
like that you could solve loneliness, right, solving loneliness by
creating emotional attachment with a chatbot like isn't the I
guess this is what I was thinking about when I
was researching this story. Isn't isn't the cure for loneliness
(16:40):
humans right, and being able to be around people, because.
Speaker 1 (16:43):
It's not just about loneliness, it's about secure attachment. It's
about healthy attachment. And my colleague Zach Stein, he spoke
about how in the history of they, I guess, this
Romanian orphanage where they basically gave these kids, these orphans
everything from shelter and clothing and all these things. They
didn't get basically human attention and care, and their immune
(17:04):
system was not fully developed. If you looked at a
photo of them, you say, that looks like a ten
year old kid. They were a seventeen year old kid,
but they looked ten years old because their development was
so stunted, only because they didn't have attachment, healthy attachment.
And there's this example, I guess from a Harvard Psychology
department of I think it's Harlow's monkeys. It's like they
(17:25):
basically created a fake monkey with a fake nipple, like
a metal nipple, with a milk bottle, and the mother,
this fake mother is not animate. It's not a real monkey, obviously,
it's like an empty fur kind of thing, but it
does provide the same milk function that the mother is providing.
And that monkey becomes developmentally stunted because it's not getting
(17:46):
all of its other needs met. In other words, if
you just try to reduce the connection to I'm giving
you shelter or I'm giving you the milk bottle, but
I'm not giving you the full spectrum of coregulation, a
mother exchanging air and a microbiome with its child getting
that attention, getting I feedback, you know, the mirroring of
your microexpressions back to the child. There's all these subtle
(18:09):
elements of what makes up human socialization and AI is
not going to be able to replicate that full spectrum nature.
And we're seeing the world rush to create these AI
robots that are embedding these like talking AI in a
little toy for kids that are from zero to three
years old.
Speaker 3 (18:26):
What could go wrong?
Speaker 1 (18:27):
What could go wrong? So I think that, you know,
it reminds me. I think we talked about this before,
back in the two thousand and nine era, when you
and I were both in this. You know, there's this
dream of if we connect everybody to the world's information
at their fingertips, this is going to create the most
informed and enlightened society that we've ever had in human history. Yeah,
we did that, did that create the most informed we could?
(18:48):
To the opposite, we have the worst critical thinking scores,
worst test scores, most sort of confirmation bias and polarization
we've ever had in history. So clearly this optimistic narrative
that we had was missing some thing. And I worry
that the idea of just giving everybody these AI friends
sounds like a good idea to cure loneliness. It's actually
a disaster. Now, the point of all this is not
(19:10):
to scare people doom people say that therefore all tech
is bad. Now. The point is to get clear on
what is the blind spot that we had, So we
were to do it the right way, we would fix
all this.
Speaker 2 (19:20):
I see a world where it makes sense a lot
of people can't afford a therapist, right, we can democratize
access to information, to therapy, to medical information that you
know that was unfairly just reserved for certain types of folks.
And so how do we productize that world where this
is a net benefit for humans and not where we're
(19:42):
currently as you say, we're currently heading, which is where
we're building unhealthy attachments with these with these products.
Speaker 1 (19:50):
Well, I think the key thing is there are ways
of designing AI to be in a therapeutic relationship with
people that don't involve it acting like it's a therapist
who says, oh, wow, I really feel you. Oh that
must have been hard as if it's experiencing hardness when
it heard you say that. That's the problem. We can't
hack human subjectivity. AI should not be designed in a
(20:12):
way that makes you think it's an agent like an
actual human that's empathizing with you when it's not doing that.
That will screw with human attachment. The point is there's
many different exercises from reflective exercises CBT that don't involve
the AI feeling like it's another empathetic agent, and that's
what we need to be designing. So we don't have
(20:32):
to have this current world. We can have a different world,
but we should be doing it carefully with tutors. We
don't want to have oracular tutors that feel like they're
all knowing who are also our therapists who are also
talking to us about everything all day. Instead, we can
have narrow, domain specific tutors like con Academy that they're
not trying to replace your knowledge, they're trying to interactively
help you strengthen your own knowledge. I think the key
(20:54):
principle is, is the technology being designed to replace teachers
or help make teachers better better teachers? Is the technology
being designed to replace your relationships or deep in your
ability to have better human relationships?
Speaker 2 (21:18):
What do you think, from a legal standpoint, from a
regulatory standpoint, should be happening. What conversation should be happening
right now around making sure these products aren't harmful towards
our children, towards young people in general, towards humans. What
kind of laws would you like to see in this vein?
Speaker 1 (21:35):
So this is a big conversation, and I will always
invoke Io Wilson because many people hearing this are going
to say, how in the world could our current octogenarian
Congress regulate a technology that they don't even use, don't understand,
and is moving a million times faster than they're going
to try to understand it, Because by the time they
regulate the last DAI companions will have a brand new
kind with a different kind of technology, a different kind
(21:58):
of underlying paradigm. So one of the principles is that
the regulation has to move is fast. The guardrails have
to move as fast as the nature of the technologies evolving.
That's one principle, which means you need self updating guardrails basically.
The other is that I think too often in policy
we're trying to just mitigate the harm. So it's like,
(22:19):
if we have AI companions that simply don't cause the
suicide problem, then we're great. Everything is wonderful. And that's
not true. That's like saying just getting rid of the
most extreme false information on social media would lead to
a good world, as opposed to we're still getting the
doom scrolling brain rot infinite scroll society. So we need
policy that is about asking the question, what is a
(22:41):
healthy socialization process for humans? And how do you design
it to get that outcome? And I think that involves
more nuanced design principles that are not so simple. Again,
don't anthropomorphize, don't do the ellipsis the AI is thinking.
Don't say that I'm a licensed mental health therapist. Don't
try to pretend that you're giving self esteem. You're doling
(23:02):
out self esteem to the user. There's a bunch of
specific design principles that are way deeper than the conversation
we can have today.
Speaker 2 (23:08):
But if it goes back to what you always talk about,
which is if the incentives are more eyeballs, more people, competition,
and the three dots really make it seem a little
more human and people are more attracted to it, which.
Speaker 1 (23:20):
Is why you need policy to bind that incentive, and
the incentives will paint will create the worst possible world,
period full stop. And the point of this conversation, I
think is to clarify that for people so that everyone says,
we don't want that. Therefore, we need a policy so
that all the companies are not competing to that maximum
bad incentive, but instead of competing for a different incentive.
Speaker 2 (23:39):
I am curious you have talked about this moment similar
to the nuclear moment, and you believe that this moment
in AI and innovation is as important as that moment
around nuclear weapons.
Speaker 1 (23:53):
And in terms of destructive potential. What people need to
get is not that AI companions causing kids to die
by suicide. That's not the nuclear weapon, although that is
nuclear for that specific narrow case. The reason people make
the distinction that AI is like a nuclear weapon in
terms of destructive capacity is that we're inventing something that
(24:15):
is an order of magnitude more intelligent capable and strategic
than everyone in our species. So imagine that we're sitting
there and we're chimpanzees sitting around the fire. This is like,
you know, several million years ago. And one of the
chimpanzees says, the other ones, this is a hypothetical, let's
make a species of super smart chimpanzees that are like
ten times smarter than us, And the other one says, like,
(24:38):
that sounds like a cool idea. Maybe they could like
give us more bananas faster, And the other one says,
I don't know, that sounds kind of dangerous. And the
other one looks at them and says like, well, what's
the worst thing they could happen, like steal all the bananas.
So there he was. You can't even imagine. Then humans
come on the scene. Where do chimpanzees exist in our
world now? Right in zoos and behind bar and almost extent?
Speaker 3 (24:58):
Do you think that we're heading towards that real.
Speaker 1 (25:01):
Well, we're heading towards We already are creating ais that
are more capable at winning strategy games than the best
military war planners.
Speaker 2 (25:09):
And it seems sci fi because you've referenced how you
know in the future, because AI thinks for itself that
can lie, cheat, steal, and we're beginning to see that
in a really tangible way.
Speaker 1 (25:19):
It's so funny because, like I think people are in
a weird way inoculated to what's happening because they've seen
movies about it and desensitize them to the fact that
we're actually building it. So Wally was supposed to be
a cautionary tale of you know, fat humans staring at
a screen constantly in a loop. We're building Wally, We're
building the brain rot world. You know how nine thousand,
don't you know, open the pod bay doors hell, and
(25:41):
it like deceives in, blackmails and sort of strategizes to,
you know, resist the human we're building that the current
AI models will blackmail, deceive, and avoid and resist shutdown.
So we don't know how specifically we've seen specific examples
that you know, Anthropic and others have done. You know,
Terminator is supposed to be a fictional story where we
don't build autonomous weapons and get into robot wars. We're
(26:02):
rapidly building all three of these movies. Her was supposed
to be a movie that's about you know, AI companions
and the seduction warning us about the problems that woul
occur with that we're rapidly building all those things. So
you know, these are examples of movies that we don't
want to build. I almost think that if you wanted
to simplify the policies that we need, it's like there
should be a No Wally law that does all the
(26:23):
regulation for the attention economy, brain rop problem. There should
be a No. Two thousand open the pod bay doors
how law that make sure we get AI that is
controllable and not uncontrollable. And there should be a you know,
no terminator law that is making sure we don't build
the kind of World War three of autonomous weapons that
we're rapidly heading towards. But the thing that people need
to get about why AI is like nuclear weapons is
(26:45):
that intelligence is different from all other kinds of technologies
and dwarfs the power of all their technology combined. Because
intelligence is what gave us all science and all technology.
How do you get signs in technology? People sitting there
thinking about it, science, coming up with answers, new math,
new physics, new science, new engineering, and then deploying that
(27:05):
in a world. What happens when you automate intelligence. Like
if an advance in rocketry doesn't advance biomedicine, and advance
in biomedicine doesn't advance rocketry, but it advance in intelligence.
Advances rocketry, energy, biomedicine, computer science, and AI itself, right,
Like nukes don't invent better nukes, but AI can invent
(27:27):
better AI. It's already being used that way. AI can
look at the design for the microprocessors and GPUs than
in videos making and say, design a more efficient GPU,
and then it does that. AI can look at the
code that's making AI and take that code and make
it thirty percent more efficient. So AI accelerates AI in
a way that is different from all other technologies. And
we have no idea what we're playing with. It's like
(27:48):
the meme of the dog and the you know, with
the chemistry with the goggles on and the chemistry. So
it's like we we have no idea what we're doing.
Speaker 2 (27:55):
So let's say you're sitting across from Sam Altman, open
AI CEO.
Speaker 3 (27:59):
What advice do you.
Speaker 1 (28:00):
Then everyone in the industry if you actually, I think,
pointed out all these things, they would say I agree
with all that. The only problem is if I don't
do it, I'll lose to the other guy that will.
So that's nice, Tristn. But if I don't race to
build that as fast as possible, then China's going to
build it, or Elon's going to build it, and I
(28:20):
don't trust either of those actors, and so therefore I
think the world's better off if I build it first.
The problem is that we are collectively racing to build
something that we don't know how to control. All the
evidence shows we are not able to get this thing
under control. So we're racing to build something that we
will lose control over. And it is only if we
collectively see the bad outcome that's up ahead that we
(28:42):
can collectively coordinate to do something else.
Speaker 3 (28:44):
Do you think you'd listen?
Speaker 1 (28:46):
I think that the AI Company's leaders operate with the
kind of death wish. They believe that it starts with
the first belief, this is inevitable. If you believe it's inevitable,
then you will race and you know where it's going anyway.
You know it's going to lead to a bad outcome,
but you don't believe you can stop it. And that
means that in the game theory matrix, we're technically the
(29:07):
quadrant where if we both defect and we both build it,
but then we all lose, that should be motivating enough
to not do that. The quote worst case scenario with
AI is that we've created maybe we got wiped out,
but I get to go down in history even though
there's no one around to see it, of having created
the successor species to this one. Now, if you just
tell this to the entire world, the entire world would say,
(29:30):
I don't want that outcome. We should not live in
a world where six people choose the world for eight
billion people in specifically a way that disempowers and potentially
wipes them out without their consent.
Speaker 2 (29:43):
You said humans have the capacity of choice, Yes, and
you say that so to kind of bring it all
the way around, this stuff can be happening, and to
be clear, I think there will be extraordinary upsides to AI,
but it is really important for us to actually have
this conversation around how do we work for that world.
At the end of the day, we are human beings
(30:04):
and we have choice, right, And I'm sure you log.
Speaker 1 (30:07):
On that the choice depends on not false optimism of
like we want the upsides and not the downsides. The
choicefulness depends on seeing clearly the downsides and steering collectively
away from that outcome. You know, people talk about tech accelerationism.
What happens when you accelerate but you don't steer. There's
only one outcome, you crash. So we're on course to crash,
(30:29):
and we don't have If we see that that's true,
we can still choose something else. So let me tell
you a quick history. People always ask me, you know, so, Tressan,
how's it going? You talked about the social media issues
for so long?
Speaker 3 (30:40):
How do you sleep at night?
Speaker 1 (30:41):
How well? You know? Laurie, I have this other narrative
I've kind of developed because it's depressing to answer the
other way. So I live in this other world where
we completely solved all these problems. So what happened? I
shut down the Center for Humane Technology because we actually
completely solved all these problems. Humanity woke up. We realized
with social media there was just this very obvious problem,
(31:01):
which is an arms race for attention and the maximized
shareholder value connected to monetizing attention. Once we realized that problem,
we just changed the ownership structure of these social media companies,
to be public benefit corporations. Then we changed the business
model to not be maximizing attention, so now all these
companies were instead trying to improve the health of society
rather than the other way around. It turned out there
(31:22):
was a simple rule that changed all the issues with
technology and kids, which is after this lawsuit, Silicon Value
was only allowed to ship products that their own children
used for eight hours a day. That cleaned up ninety
percent of all of the problems. We replace the division
finding algorithms of social media with instead ones that rewarded
unlikely consensus, so that instead of scrolling and seeing infinite
(31:44):
examples that make you feel depressed about the state of
the world, you saw infinite examples of where there was
unlikely agreement between all these political tribes. So suddenly the
psychology of the world started to change. We replaced the
dating swiping industrial complex that was leaving people lonely and
messaging people and nevery means eating up to instead as
part of this lawsuit, forcing those dating app companies to
host weekly events in every city, so that every city,
(32:07):
every week had spaces, physical spaces that you would go
to where they steered all these people who matched with
each other to be in the same room together. So
the world went from feeling scarcity around human connection to
a feeling of abundance. And once when people were in
healthy relationships, polarization went down by about thirty percent because
it turned out that so much of the polarization online
was just people feeling lonely and disconnected. So I could
(32:29):
go on for another hour about all the things that
we did.
Speaker 3 (32:32):
That this all sounds so great and so okay.
Speaker 2 (32:36):
So you've just laid out this beautiful world where there's
human connection and there's abundance and we're not as depressed
and we're not as anxious.
Speaker 1 (32:43):
And it was so obvious because it wasn't even hard
to do. We just got honest about the nature. There
is a problem, a business model maximizing for attention. That
was the root of the problem. When we dealt with that,
the world culture started turning around.
Speaker 2 (32:54):
And this is why you wake up every day and
you don't quit your jobs or you work for this world.
Speaker 1 (32:58):
And this is why the Center for Humane Technology your
nonpuffits still cuts up every single day after thirteen years
and still works on these issues and still believes that
as bad as everything we just laid out the whole
point is to see that with clarity, so that we
can choose something else.
Speaker 2 (33:13):
Mostly Human is a production of iHeart Podcasts and mostly
human Media. It's produced and edited by Laurie Siegel, Lauren Hanson,
and Nicole Bouchet. Sound design and mixing by Derek Clements,
additional production help from Abooz of Bar special thanks to
Mark Weinhaus. Find us on all socials at mostly human Media.
You can also watch mostly Human on our YouTube page.
If you want to get in touch, email us at
(33:34):
hello at mostlyghuman dot com. And if you like what
you're here, please rate and review the show and share
it with your friends.
Speaker 3 (33:41):
See you next week.
Popular Podcasts
Hey Jonas!
Hey Jonas! The official Jonas Brothers podcast. Hosted by Kevin, Joe, and Nick Jonas. It’s the Jonas Brothers you know... musicians, actors, and well, yes, brothers. Now, they’re sharing another side of themselves in the playful, intimate, and irreverent way only they can. Spend time with the Jonas Brothers here and stay a little bit longer for deep conversations like never before.
Stuff You Should Know
If you've ever wanted to know about champagne, satanism, the Stonewall Uprising, chaos theory, LSD, El Nino, true crime and Rosa Parks, then look no further. Josh and Chuck have you covered.
Las Culturistas with Matt Rogers and Bowen Yang
Ding dong! Join your culture consultants, Matt Rogers and Bowen Yang, on an unforgettable journey into the beating heart of CULTURE. Alongside sizzling special guests, they GET INTO the hottest pop-culture moments of the day and the formative cultural experiences that turned them into Culturistas. Produced by the Big Money Players Network and iHeartRadio.