May 21, 2026 • 50 mins
In this episode of News Items, John Ellis sits down with author and Council on Foreign Relations fellow Sebastian Mallaby to unpack the astonishing rise of Demis Hassabis and the race to build artificial general intelligence. From Hassabis’s childhood as a chess prodigy to the creation of Google DeepMind, the conversation traces how a London-born gamer and coder became one of the most consequential scientists alive. Mallaby explains the breakthroughs behind AlphaGo and AlphaFold, why AI may soon revolutionize medicine faster than the human genome project transformed biology, and why the technology’s dangers are no longer theoretical. The discussion moves from Hassabi’s Nobel Prizes to rogue states, AI arms races, cyberwarfare, and the unsettling possibility that machines may soon improve themselves faster than humans can understand them. It is a fascinating, occasionally terrifying portrait of a future arriving far sooner than most people realize.
Hosted by John Ellis
Listen
Watch
Mark as Played
Transcript
Episode Transcript
Available transcripts are automatically generated. Complete accuracy is not guaranteed.
SPEAKER_00 (00:00):
Hello and welcome back to the News Items Podcast.
I'm John Ellis.
I'm the founder and editor oftwo Substack newsletters.
One is called News Items, theother is called Political News
Items, and you can find themboth at news-items.com.
Our guest today is SebastianMalaby.
Sebastian is the Paul A.
(00:20):
Volcker Senior Fellow forInternational Economics at the
Council on Foreign Relations, atwo-time Pulitzer Prize
finalist.
He's the author of six books,including More Money Than God,
Hedge Funds and the Making of aNew Elite, and The Power Law,
Venture Capital and the Makingof the New Future.
Sebastian, thank you very muchfor taking the time to be with
(00:42):
us today.
We're here to talk about yourlatest book about Demus
Hassabas, I think I have itpronounced right, finally.
SPEAKER_01 (00:49):
Right.
SPEAKER_00 (00:50):
And I think it's safe to say that Mr.
Hassabas is doing or is one ofthe very few people who is doing
uh the most important work onthe planet.
And before we get to the mostimportant work on the planet, I
thought it'd be helpful to thelisteners if you'd give us a
brief sketch of how Mr.
Hassabas got from being a childchess prodigy in his preteen
(01:13):
years to being the now chiefexecutive of Google Deep Mind, I
think the chief scientist andand the winner of the Nobel
Prize for Chemistry in 2024.
SPEAKER_02 (01:24):
Sure.
Well, Demis Hassabis was born toimmigrant parents in London.
His father was of Greek Cypriotorigin, his mother, Chinese
Singaporean, spent some time asan orphan on the streets of
Singapore as a child.
I like emphasizing this toAmerican and particularly
Silicon Valley audiences becauseit just gets across the point
that maybe Silicon Valley doesnot have a monopoly on melting
(01:47):
pot features or kind ofimmigrant founders and so forth.
Demis, you know, had this modestbackground, but his talents
proved to be immodest.
And when he saw his fatherplaying chess against his uncle,
aged about four, he just sort ofwatched the game a bit and then
he understood it, and then hecould beat adults quite soon
after that, and then he was thebest junior chess player in
(02:07):
Britain and second best in theworld, and pretty much expected
to become a chess professional.
But along the way, he startedwinning money from winning chess
tournaments, and he would buyhimself computers, which he
would upgrade when he got somemore money.
And he got into coding, and thesegue was actually reading a
book about chess programming sothat he could marry his interest
(02:28):
in computers with his interestin chess.
And the ideas in this chessprogramming manual kind of went
back to information theory, toTuring's early ideas, what is
computing?
And it, you know, struck him.
And a little bit later, he foundhimself, you know, he'd been
given a position to study atCambridge University when he was
extremely young.
And the university told him,listen, you're academically
(02:51):
ready, but you shouldn't comeyet because you're not socially
ready.
I think he was about 16 or so.
Um so go do something else for abit.
So he went off and worked for avideo game design company using
his coding skills.
And in this community of kind ofslightly autodidact misfits who
were building video gamestogether, is this sort of early
1990s?
He would be discussing how youcould make these games more
(03:14):
enthralling and interesting.
And that included discussions ofartificial intelligence.
The idea that, you know, theplayer of the game could treat
characters in the game incertain ways that would
encourage them to do more ofsomething or less of something
else.
This would be a basicreinforcement model, as people
calling it now in AI.
So ideas about AI started topercolate when he was around 17.
(03:38):
He read this book,Gürtel-Escherbach, which,
amongst many things it says,posits the notion that, you
know, basically the human brainruns on ones and zeros.
And so one day a computer with alot of ones and zeros and a big
enough architecture could dowhatever the human brain could
do.
And so putting this together, bythe time he was going to
Cambridge at the age of around18, Demis had formed this
(04:02):
ambition to create superpowerfulAI, which is just an astonishing
fact in itself.
I mean, this is somebody thatyoung and having this view in
1994, when we are kind of 18years before AI being able to do
a single thing.
I mean, it couldn't recognizethe photograph of a cat until
2012.
(04:22):
And so this early conviction ispart of what makes Demian such a
fascinating character.
SPEAKER_00 (04:27):
How did you come to get the access that you did?
I believe you had 30 interviewswith Mr.
Hosabas over uh you know anumber of years.
Did he approach you?
Did you approach him?
How did it work?
SPEAKER_02 (04:40):
Aaron Ross Powell I approached him.
Um I had met him um at techconferences because my previous
book was about uh venturecapital investing, and so I was
interested in technology.
I would go to tech conferencesin Europe and there would be
this sort of unlikely figure whowould show up looking about 25
years old, although he wasprobably more like 35 or 40 at
(05:01):
the time.
And uh and he would, you know,have this approachable, boyish
demeanor, extremelyunpretentious, kind of like the
kid next door who says, Hey,let's get a sandwich and uh
we'll do the dishes after lunchand then go for a walk.
How does that sound?
I mean, just super, superrelatable.
And then he would get on thestage and start talking, and
ideas about neuroscience andcomputer science and biology and
(05:24):
chemistry and physics and thehistory of movies and philosophy
would just tumble out of hismouth in this astonishing
stream.
So the juxtaposition between theapproachability of the manner
and the ambition of the ideasalways grabbed me.
And then around about 2022, whenI was thinking about the next
project after writing aboutventure capital, it struck me
(05:45):
that AI was sort of ripe.
I mean, ChatGPT was not out yet,but you'd seen examples of
powerful AI in the protein shapeprediction project that Demis
Susabus had done with GoogleDeepMind.
So I felt that the technologywas ripe, and this was an
amazing character.
And if I could get him to agreeto talk to me, I would be able
(06:06):
to tell the story of the makingof modern AI through this
unbelievable personality.
So I went to pitch him, and thepitch was basically: look, I've
listened to all your lectures,Demis.
And in these lectures, you tellus that the invention of
artificial intelligence is goingto be the most consequential
invention of human history.
Now, if that's true, then itfollows that you're going to be
(06:26):
one of the most important peoplein human history.
So, Demis, you don't have achoice.
There will be books about you.
Let's just get real here.
Don't pretend that you can hide.
And furthermore, you should wantthere to be books because you
cannot unleash a technology thisdisruptive on the world, which
is going to make people raisetheir children differently, do
their jobs differently, conceiveof themselves as humans
(06:47):
differently, and not explain whyyou're doing it.
I mean, it's dangerous, it'sdisruptive.
You have to say what your valuesare, what your motives are, what
the point is.
And so you should welcomesomebody who wants, who comes in
the front door, not the backdoor, and you know, promises to
take your ideas seriously and todo a good, you know, careful,
multi-year project where you Ireally tell your story.
(07:09):
And he thought about that for abit and he accepted.
And then right around when heaccepted, AGPT came out.
So my fringe topic went to themainstream faster than I could
have imagined.
SPEAKER_00 (07:19):
Aaron Powell A famous historian once said it's
it's almost impossible to writea biography about someone you
don't like.
Only a good biography uh can bewritten by can be written about
someone that you do like.
Did you like him?
Do you like him?
SPEAKER_02 (07:34):
I do like him.
One of the interesting thingsactually is that even somebody
you massively respect and likeis going to have blind spots.
I mean, I think of this asrelationship advice as well as
just sort of biographer advice.
And with Demis, you know, sureenough, there were certain
things where he was not able tosee himself lucidly.
Uh so for example, he alwayssays he doesn't like to control
(07:57):
people, that his mother broughthim up with these Christian
values not to control people, torespect everybody.
And it's true that he he carrieshimself in a way that's you know
designed to be non-intimidating.
And I have funny stories about,you know, because I live in
London, Demis Osavis lives inLondon.
It turns out that I have afriend whose kids were at the
same school as Demis' kids, andthis friend of mine went over to
(08:20):
Demis's house to pick up his sonfrom a birthday party, and he
saw Demis and he recognized himfrom a YouTube video.
So he said, Oh, you're the AIguy.
And uh Demis said, Yeah, I'm theAI guy.
And my friend said to Demis, um,you know, I I I watched this
video about AI on YouTube, andwhat it strikes me is that you
could make a lot of progress ifyou had two AIs and they kind of
(08:42):
argued with each other and thesynthesis of their views might
be better than either one.
And so now you have the, youknow, totally non-technical
random friend of mine who becamea publisher after leaving
university lecturing the futureNobel laureate on what he should
do with his AI models.
So I said to my friend, look, sohow did Demis respond to your
useful advice on the future ofAI?
(09:02):
And my friend said, Well, Demissaid it was a very useful
insight.
So he is a nice guy, right?
Well, I know, I know in thiscompletely unplanned piece of
reporting that he is a reallynice guy.
But he says he doesn't controlpeople, which is ridiculous
because he runs a large companyand he's smarter than everybody
else and more articulate, and hewins 99% of the debates he has
(09:25):
with people.
So of course he controls them.
But anyway, he refused to seethat.
And we had an argument aboutthat.
SPEAKER_00 (09:30):
There are two sort of, it seems to me, two
milestone moments in the historyof Google.
One was the acquisition ofJeffrey Hinton's company, which
was really like a two-page memo.
But Mr.
Hinton is often called thegodfather of AI, and Google
acquired his brain, essentially,and he worked there for a number
(09:51):
of years.
And the second, obviously, is uhGoogle's acquisition of
DeepMind.
How did that come up that comeabout, I guess, is the question.
SPEAKER_02 (09:59):
Yeah, so Demis founded DeepMind in 2010.
And um, for the first three anda bit years, it was sort of
raising money from Peter Thielin Silicon Valley, raising money
from a few other investors.
Elon Musk put five milliondollars in, was a Singaporean,
uh Hong Kong-based investor aswell.
(10:21):
So there's, you know, it waskind of gathering momentum, but
it was a struggle to raiseenough money to afford enough
computing power to really take abig leap forward.
And then Google becameinterested in buying it, partly
because they had already boughtthe Jeff Hinton boutique you
mentioned.
And Jeff Hinton had recommendedbuying a Demis company because
he knew Demis, he'd met Demis,he knew Demis was extraordinary.
(10:43):
And so, based on thatrecommendation and on a model
that Demis' company DeepMind hadproduced after a couple of
years, which was the firstagentic games playing model.
It was a model that could justthrough trial and error, without
being given the rules,understand those Atari video
games from the 70s and 80s,games like Pong and Breakout and
(11:05):
Sequest and stuff like that.
So Owl was sufficientlyimpressed by the technological
progress and impressed byHinton's recommendation and
wanted to buy DeepMind.
And Demis, you know, was keenbecause he wanted to have
liberation from the hamsterwheel of fundraising from the
venture capitalists.
(11:25):
And in fact, Larry Page, um,they met at an Elon Musk
birthday party, but it's fun,these stories are fun because it
shows you how all of thesepeople were entangled, you know,
right at the start, you know, inthe 2010-2012 zone, the same
characters were circling aroundeach other, kind of friendly but
competitive, sometimes viciouslycompetitive, as we saw recently
(11:46):
in the in the Musk v.
Altman trial.
But anyway, um there was thisbirthday party of Elon Musk's,
and Larry Page suggested goingfor a walk with Demis.
And in this walk, he said toDemis, look, you have to decide.
You could probably build acompany as big as Google if you
want, but it will take you allyour energy.
If what you really are is ascientist, then you should come
(12:08):
and do your science with thebenefit of my resources and my
computing power.
And to Demis, that was ano-brainer.
He was much less interested inbecoming a multi-billionaire
than he was in being the personwho delivered powerful AI to the
world.
SPEAKER_00 (12:22):
Aaron Ross Powell And with the acquisition by
Google, he he becamesubstantially wealthy.
So he didn't even have to worryabout that anymore.
SPEAKER_02 (12:30):
Aaron Ross Powell Yes.
Um to any normal person, walkingoff with a hundred and something
million is extremely wealthy.
It just wasn't on the scale thatuh you know Larry Page composed.
SPEAKER_00 (12:40):
Aaron Powell But you get the sense from reading your
book that he doesn't he doesn'treally care.
I mean, obviously he caresbecause it makes it makes life
easier and he doesn't have tofly coach or whatever.
I mean, is it is is that right?
It does he really is that asecondary concern or even a
tertiary concern for him?
SPEAKER_02 (12:57):
Aaron Powell Yeah.
I mean, I he he would always sayto me he doesn't care about
money.
So one day I quizzed him, youknow, we would meet in this pub
near his home in North London,and at the back of the pub there
was a dusty staircase.
You'd go up there and there'd bea abandoned room which nobody
really used.
And so we'd get coffees and wesit up there for two hours at a
time.
(13:17):
And in one of these sessions Isaid, okay, you say you don't b
believe in you don't care aboutmoney, so let's just go down the
list here.
What kind of car does yourfamily drive?
And he said, Oh, I've forgottenwhat it's called, but I think
it's 11 years old or something.
Do you have a second home?
No.
Do you have a yacht?
Of course not.
Do you collect really expensiveart?
No.
Well, do you have any hobbies?
Yeah, I do actually have ahobby, it's true.
What's the hobby?
Well, I I really supportLiverpool Football Club.
(13:41):
And I did buy myself a couple ofseason tickets.
I said, How much does that cost?
And he said£3,000, which is like5,000 bucks.
And that's kind of the size ofit.
I mean, you know, perhaps thereare some things that he didn't
tell me.
I think I'd find out later thathe had done a fairly fancy
renovation to his home.
But still, it we're talking, youknow, a a very nice middle class
home, not a kind of crazymansion.
(14:03):
And uh yeah, so I don't think hecares about money.
SPEAKER_00 (14:05):
So it's all about the work.
SPEAKER_02 (14:06):
It's about science, it's about ego, it's about fame,
it's about a sort of messianicdesire to be the person who
brings artificial intelligenceinto the world.
And then beneath that, or behindthat, there is something else.
And this surprised me.
Uh there was one time we weremeeting, and instead of being
inside in the pub, it was such anice day, even, you know, even
in London, you get a few nicedays.
(14:28):
So we went out to a park and wesat in a cafe, and the people at
the next table were havingnormal cafe type conversations.
You know, my friend was sick,she went to hospital, oh, I hope
she's better, etc.
And I was sitting oppositeDemis, who was in this sort of
messianic riff.
And he started saying, at two inthe morning, Sebastian, when I'm
reading a scientific paper, Ifeel that reality is staring at
(14:50):
me in the face, calling to me,waiting to be figured out.
We have to figure it out.
It's absurd.
We don't know what gravity is,we don't know what time is.
There are so many mysteries inphysics which you haven't yet
figured out.
I mean, this table, Sebastian.
And then he banged his fist onthe table.
He said, It's made of atomsjumping around with spaces in
between.
Why is it solid?
We don't even understand that.
We need to understand thesethings, and that's why I'm
(15:11):
building AI, so that we cancrack these scientific mysteries
and understand the true fabricof reality.
And that is what I need to dobefore I die.
And if I can understand hownature functions, I will be
getting closer to theintelligent being that may
presumably have set upeverything in reality in a way
that is amenable for humans togo and do science and to
(15:32):
understand it.
I will get closer to thatintelligent design, which could
be my definition of God.
SPEAKER_00 (15:37):
That's the way I feel when I wake up in the
morning, you know.
Slightly different.
I think we have to talk aboutone of the major breakthroughs
that generated a ton ofheadlines, which was the
artificial intelligencedeveloped by Mr.
Hashabus that defeated theworld's greatest Go player.
(15:59):
Can you tell us that particularstory?
SPEAKER_02 (16:02):
Sure.
So after Google bought DeepMindin 2014, Demis went off to a
conference with the seniorleadership of Google.
So there he is with his newproprietors.
And one of them, Sergey Brynn,is a Go player.
And Demis, you know, drops acomment to the effect that
maybe, you know, an AI systemcould defeat a Go player.
(16:24):
And Sergei Brynn, who knewsomething about computing and
something about Go, said, no,no, that's ridiculous.
Surely that's impossible.
And that was the standard viewat the time, that the idea that
you could have an AI that waspowerful enough to solve the
incredibly intricate patternsthat you get in Go and to derive
meaning from them when thecombinatorial complexity of how
those, you know, all thepermutations of how those stones
(16:47):
could be placed on the board,they're just, you know, almost
infinitely large.
It's a 19 by 19 board.
So the very first move, there's361 possibilities, second move
360, next one 359, you multiplythat out, you very quickly get
to this insanely big number.
So Sergei Brynn was justchanneling the consensus when he
said, you can't possibly dothat.
And Demis said to himselfsilently, if he thinks it's
(17:10):
impossible, then it ought toimpress him if I do it.
And then, you know, we'll getall the funding we ever want.
So he went back to his friendDavid Silver, who was one of the
scientists at DeepMind and afriend of Demis's from
University of Cambridge whenthey both did computer science
together.
And he said, um, so David, youyou you did your PhD on um
primitive go-playing AI.
(17:32):
Do you think you could make oneactually that really worked?
And Dave Silver's view was yes,because there had been advances
between him finishing his PhDand the moment when Demis asked
him this question.
There had been adequate progressin both deep learning and
reinforcement learning, whichare the two main strands of
artificial intelligence, that ifyou combine the two with some
(17:53):
novel tactics, this ought to bea soluble problem.
So Demis said, great, we know,let's do it.
And um he gave him, gave DavidSilver the resources, backed
him, and it took about a coupleof years.
But in this famous game in thespring of 2016, Deep Mind sort
of staged a new version of thatdeep blue versus Gary Kasparov
(18:15):
moment for chess.
Right.
And this instead was Alpha Goversus Lisa Dole.
And Lisa Dole is a uh SouthKorean world champion.
And uh it was best of five.
There was a prize of a millionbucks for the winner, and um
AlphaGo won four games to oneagainst the human.
And to this day, when Demis goesto South Korea, he's mobbed by
(18:38):
people as the father of AlphaGo.
And I heard on his most recentvisit, you know, people rushed
up to him and demanded that hesign a copy of my book, which is
the moment I think when finallyDemis accepted that he was happy
that I wrote the book.
SPEAKER_00 (18:50):
So AlphaGo, if we I mean, we can there are a lot of
steps along the way, but if wego from Alpha Go to Alpha Fold,
can you inform the listeners ofwhat Alpha Fold is and why it's
so important?
SPEAKER_02 (19:05):
Aaron Powell Sure.
So after AlphaGo was finished,now we're in the spring of 2016,
and Demis is walking out of thesort of exhibition hall where
the game took place with hisfriend David Silver, and it
happens that behind them, asthey walk down the street, there
is a camera crew with a bigmicrophone, sort of one of those
kind of fuzzy wooden things thatthey stick out at you.
(19:27):
And it picks up the conversationbetween Demis and Dave.
And unbelievably, Demis issaying, you know, having just
collected the laurels of this gobreakthrough kind of 20 minutes
earlier, he's saying, right,Dave, now we can do protein
folding.
And so, you know, he rested onhis laurels for kind of 15
minutes and announced the nextchallenge.
(19:48):
And this next challenge issomething that went back to his
time at Cambridge, where he hada biologist friend who told him
about a Nobel laureate inbiology called Christian
Anfinson, who had delivered aNobel lecture in which he had
conjectured that if you take anamino acid strand and you look
at the DNA sequence on thestrand, the sequence is a sort
(20:10):
of code.
And if you could read the code,you would predict how that
strand of amino acid would folditself up into a beautiful,
intricate shape.
And this is the shape of theprotein that it forms.
So proteins are these sort ofmini building blocks in nature,
in our bodies, in plants,everywhere.
Um, and they're they're composedof these strands of amino acid
(20:32):
that kind of like aself-executing origami model
twist themselves into anintricate shape.
And so this was anextraordinarily ambitious idea,
right?
That you could predict a shapewith actually more permutations
in the way that that strand ofamino acid could fold itself up,
more permutations than even inGo.
But also is much moreconsequential than Go, because
if you could understand all theshapes of all the proteins in
(20:56):
nature, you could in turn designor choose molecules that would
bind onto these proteins if youwere trying to build a medicine,
or possibly if you were tryingto do some innovation in
material sciences and a bunch ofother applications.
And so solving this problem wassimilar to Go in the sense that
it was a big combinatorial spacewith almost infinity of
(21:21):
permutations.
And so that's why Demis, havingconquered Go, immediately
thought of protein folding next.
But it was even harder and moreconsequential.
So anyway, after that, Demis uhconvened a team to sort of focus
on this protein foldingchallenge.
It was a long and winding road.
This one took, yeah, just overfour years.
(21:41):
But at the end of 2020, he had asystem that could accurately
predict all these protein shapesjust by knowing the DNA
sequence.
And the practical consequencewas that it used to take a PhD
researcher about four or fiveyears sometimes to do the
intricate kind of X-raycrystallography, which is the
process of.
Physically peering at thisminute protein shape and mapping
(22:04):
it.
It's so difficult.
It would take four or fiveyears.
Now, a scientist can simply lookup the protein shape like a
Google search because all of theshapes have been discovered by
AlphaFold, Demis's program, andhe has open sourced them so
anybody anywhere in the worldwho is interested in structural
biology can look up all of theshapes in nature pretty much.
(22:24):
And this is the achievement forwhich Demis shared the Nobel
Prize in Chemistry in 2024.
SPEAKER_00 (22:30):
Did Alpha Fold then morph into isomorphic labs?
Is that what happened?
SPEAKER_02 (22:35):
Correct.
After open sourcing all theshapes of all the proteins and
kind of gifting that to science,the commercial side of Google
kicked in, maybe notunreasonably, and they decided
to persist in that line ofresearch, but in a company that
would be separate from DeepMind.
It's called Isomorphic Labs.
And they would keep theirdiscoveries proprietary.
(22:57):
So where we are today is thatwe've gone from AlphaFold 2,
which was the system thatunraveled all those protein
shapes in 2020.
And now we're on Alpha Fold 4,which is really sort of a suite
of AI products whichcollectively are designed to
bring AI into the drug discoveryprocess.
And I think also perhaps in notonly to the discovery process,
(23:20):
into the clinical trial processthat follows.
And the notion is that you couldspeed up the discovery and the
productization of new medicines,you know, maybe by an order of
magnitude.
Trevor Burrus, Jr.
SPEAKER_00 (23:32):
And this is, I mean, it's all part of the story where
you marry a new technology toscience.
So and the you know, in themapping of the human genome, you
you took uh the the basicscience of genomics and you
married it to supercomputing,and lo and behold, Craig uh
Venter gets us a rough draft ofthe human genome, what, I don't
(23:55):
know, ten or fifteen yearssooner than NIH was doing it by
hand, so to speak.
Here you're married now marryingcode with artificial
intelligence, and so leaps andbounds of discovery occur.
What what is happening in theisomorphic lab space?
Are they have they come out withuh a pharma product or or a new
(24:18):
way of looking at a medicalproblem?
Is there is there a commercialapplication yet?
SPEAKER_02 (24:22):
Well, again, it i the inventions at isomorphic are
proprietary.
They don't talk about them indetail.
But but I can give you a coupleof shed light a little bit.
I sat next to a dinner recently,the co-founder of BioNTech,
which was the Germanpharmaceutical company that,
partnering with Pfizer, producedthe first mRNA COVID vaccine.
(24:46):
What she told me, uh thisscientist, is that in the drug
discovery phase, there arecertain uh you know the the the
the process from kind of havingthe concept of a medicine that
you're looking for and actuallyfinding what seems to be a
plausible molecule that works.
In some cases that's reduced toone-tenth of the previous time,
is what she told me.
And of course, you know, thereare caveats, it depends slightly
(25:08):
on what type of medicine you'relooking for.
This protein folding technologyis more useful in some cases
than in others, but it really isa meaningful speed-up.
And I've met otherpharmaceutical researchers who
say, yes, you know, I use itevery week.
It's it's really, you know, it'sa great shortcut for research.
The challenge comes in gettingthrough the clinical trials,
(25:30):
because you know, so far it'sbeen impossible to persuade any
regulator to contemplateshrinking and the time that
takes.
There is a sort of starter idea,which to me makes perfect sense
of how you could shrink it,which is that in a clinical
trial, you have both a testgroup, which get the new therapy
that you've just developed, andthen you have a control group,
(25:51):
which is supposed to bedemographically as close as
possible to the test group, butthey just get the
state-of-the-art therapy,whatever that happens to be.
But of course, because it's thestate-of-art therapy, by
definition, that means thatmany, many, many patients in
advanced Western countries havereceived the therapy already.
And if you use AI to search outamongst those people who've
(26:12):
already been treated, a groupwhich has the same demographic
characteristics as the people inyour trial group, you're in
business.
You can just simulate what theoutcome is from the
state-of-the-art therapy withoutneeding to have a whole trial
for that.
And this would be one way ofshrinking the time it takes to
get through clinical trials.
And I suspect there are morethat could be as regulators
(26:33):
develop more confidence in theaccuracy of simulated AI testing
results, perhaps they will beless exigent in the kind of
actual life human testing that'srequired.
SPEAKER_00 (26:44):
Aaron Powell Aside from birthing isomorphic labs
and another company calledIneffable Intelligence.
What by the way, what isineffable ineffable
intelligence?
What does that company do?
SPEAKER_02 (26:56):
Aaron Ross Powell So that company is founded by David
Silver, the person I wasreferencing earlier, who uh was
the genius behind AlphaGo, uhalso did a system called
AlphaZero after that.
And his particular specialsource is that he is the master
of what's called reinforcementlearning.
And and and maybe to justexplain this for a second,
because I think it's superuseful to people who understand,
(27:19):
okay, I I I know there's thisthing called AI, but what is it
under the hood?
And I think just one level down,just to give people one more
step in understanding, there aretwo main types of artificial
intelligence.
There is deep learning, whereyou have just masses and masses
of data, and you have algorithmsthat can find patterns in the
data and induce predictions fromthat.
(27:40):
It's a bit like, you know, yousend a human being to the
library and you have them readall of the books in the library,
and they're smart enough thatthey remember all the books, and
they can start thinking acrossall this crystallized human
knowledge which they've read.
Then there's a different kind oflearning, reinforcement
learning, where you're in anenvironment, and for humans it's
the real world, and you take anaction, and then something
(28:02):
happens in response to youraction, and you learn through
trial and error how the worldworks.
So you know that if you pick aglass of water up, you can feel
the weight, because there'sgravity.
Okay, now you understandsomething.
You know that if you turn theglass upside down, the water
will splash on the floor.
All right, now you understandthe difference between a solid
and a liquid.
You know that if you drop theglass, it shatters on a concrete
floor, et cetera, et cetera.
(28:23):
So as you as you take actions inthe world, you learn from the
world.
And that is the notion behindthis other strand of AI called
reinforcement learning.
There were two sort of warringtribes in artificial
intelligence in the twent in the2000s.
Deep learning was, you know, theheadquarters was Toronto, Jeff
(28:43):
Hinton's lab.
And then in Edmonton, Alberta,also Canada, funnily enough,
there was another Brit.
Jeff Hinton is one Brit.
The other one was called RichSutton, and he was the guru of
reinforcement learning, and thatis where David Silver did his
PhD, learning from Rich Sutton.
So David Silver brings thisreinforcement learning approach
to DeepMind.
He has amazing successes inbuilding these game playing
(29:05):
agents.
And now what he's doing withIneffable is trying to take
agentic AI, reinforcementlearning-based AI, and push that
to the nth degree.
I mean, he's he's although Ihaven't talked to him since he
started the company, I talked tohim an awful lot from my book.
And I know what his vision waswhen I was talking to him at the
end of 2025 and why, to him,reinforcement learning was being
(29:29):
under-emphasized by the biglabs.
And he feels as though when youwhen you learn from data, as
deep learning does, bydefinition, you can only learn
from what's in the data.
The data is created by humans inone way or another.
And so that's a limiting factoron how clever AI can be.
If you want the AI to surpasshumans, to be really super
intelligent, it has to learn byessentially creating its own
(29:51):
data through trial and error.
So in Go or Chess, the best,best systems are ones that don't
study expert human games.
They learn purely by playinggames against each other and
learning through trial and errorwhich strategies in chess or Go
work better.
And David Silver wants to extendthat idea.
Like, for example, in medicine,you could start by training an
(30:12):
AI on all the medical journalarticles.
And in this way, it would inhalethe state-of-the-art human
understanding of medicine.
Or you could say, I know what,we've got these humans wandering
around with, you know, bootbands on their wrists or aura
rings on their fingers,collecting actual data on the
(30:33):
human.
And, you know, we could get someother data on the human in
various other ways.
And we know, you know, the heartrate, the sleep, and so forth
and so forth.
And now we're going to correlatethat with health outcomes.
And so we're just going tobypass what human doctors think
they know.
We're going to go directly tothe data, and agents are going
to find patterns.
(30:55):
They're going to sort of runlittle experiments and figure
out what really is drivinghealth outcomes.
Leapfrogging over crystallizedhuman wisdom, which may not be
as perfect as we think it is.
SPEAKER_00 (31:06):
Aaron Powell One of the things that's interesting to
me is in any situation where youhave uh an extraordinary talent,
it attracts other extraordinarytalent.
And Damas's company and hiscohort have produced any number
of extraordinary talentedpeople.
Who are the most remarkable, inyour view, that have sort of
(31:28):
spun out of DeepMind?
SPEAKER_02 (31:30):
I think David Silver is a standout.
And beyond that, there have beena number of others.
The founder, the CEO of Mistral,the French AI lab, Arthur
Mensch, is another alum ofDeepMind.
And there are probably someothers who I'm not thinking of,
but one of the characteristicsof this field is that you do
(31:52):
need a ton of money to train AIsystems.
So oftentimes when somebodyleaves DeepMind, they go work
for one of the other Frontierlabs as opposed to doing a
startup.
Although increasingly, if you'refamous enough, you can raise
crazy amounts of capital, andthat gets to a whole other
discussion about bubbles and soforth.
SPEAKER_00 (32:09):
Aaron Ross Powell Well that's the subject of your
previous book, so we'll we'llstick on this one.
We have to we started by sayingit's the most important work on
the planet, or at least, in inmy opinion, the most important
uh work on the planet.
What exactly is artificialgeneral intelligence?
SPEAKER_02 (32:26):
Aaron Ross Powell There are two debates in this
field which are impossible toresolve because people don't
agree on definitions.
One is the one you just asked,what is artificial general
intelligence?
The other one is can a computerbe conscious?
Because it depends on what yourdefinition of consciousness is.
But let me just answer yourquestion on artificial general
intelligence.
You could say, look, if you useGemini or Chat GPT or Claude
(32:48):
today, it's artificial, it'sgeneral, it can talk about a lot
of things, and it's intelligent.
We have it, right?
I mean, that would be aplausible definition, a literal
definition of AGI.
At the other extreme, you havesomeone like Demis who tends to
want to push the moment when wedeclare that we have AGI.
He wants to push that out intothe future because he's having
(33:08):
such a great time, you know,doing the research to try to get
to it.
He doesn't want the research tostop, but then he wants to
extend the journey.
And then maybe there's a littlebit also he doesn't want to
freak people out by saying, youknow, we now have this
scary-sounding thing.
So for whichever reason, helikes to say AGI is further in
the future.
And in order to back up hisfurther in the future point of
(33:30):
view, he chooses a definitionwhich is very stringent.
He says, okay, let's imagine wetrained an AI system on
everything that was known upuntil 1911, and then we just
waited and saw if the AI systemcame up with general relativity
all by itself.
It's a pretty tough hurdle.
SPEAKER_00 (33:47):
I would say so.
I wanted to ask you, and JeffHinton is coming out, I think
this fall, with a manifestoabout the dangers of AI.
How does how does Mr.
Hashabas, how does Demis viewthe perils of AI?
SPEAKER_02 (34:06):
Aaron Ross Powell I mean what he says is, and I kind
of agree with him, is that look,there's a non-zero risk, that's
his phrase, non-zero risk thatthese models will start to
attack humans.
And I actually used to thinkthis was a crazy notion, and I
thought, well, of course,machines would be more
intelligent than people.
We already see that in domainslike Go, and so why wouldn't it
(34:27):
happen more generally?
But machines wouldn't attackhumans because they're not
really motivated to do so,right?
We are evolved to want tosurvive, to pass on our DNA.
Machines don't have DNA, theydon't care about survival, so
therefore, why would they attackus?
That was my kind of comfortingworldview.
And then one day I went toToronto to see uh Jeff Hinton,
(34:48):
and I sat in his kitchen for acouple of hours, and I said,
Look, Jeff, you know, surelyyou're exaggerating.
Why would these systems attackus?
They don't want to survive.
And he said, Okay, we'll do thisthought experiment, Sebastian.
Imagine you have a very powerfulAI, and you're worried that an
enemy AI, maybe a Chinese AI ora Russian AI, is going to attack
your AI.
(35:09):
You're a slow-thinking human, soyou're way too dumb to respond
to a cyber attack at digitalspeed.
So you're going to tell your AI,listen, if you see the attack
coming, defend yourself.
Maybe counterattack.
Whatever you do, survive.
Survive.
We just used that word.
Now are you feeling comforted,Sebastian?
And I kind of see his point.
I mean, I think we will endowthese systems with a desire to
(35:33):
survive.
And indeed, that desire may sortof just emerge out of the
training data because thesesystems are trained on, you
know, novels and books about thehuman condition and all sorts of
stories about humans fightingfor survival.
And so why wouldn't they get theidea of survival?
And if you marry supremeintelligence with the urge to
(35:54):
survive, with an ability to beduplicitous, which is something
that we observe in the models,it strikes me that, yeah, they
could turn against us.
Now, of course, there's a wholebranch of AI research called
alignment research, which isprecisely devoted to avoiding
that outcome.
And, you know, very smart peopleare engaged in that.
So I hope that we can containthis risk.
(36:15):
I don't put this risk at even1%.
I think it's below that.
But I think non-zero is theright answer, and anybody who
claims that there's no risk orthat the risk is 50%, um, are
reflecting something about theirown psychology as opposed to a
sober reading of the evidence.
SPEAKER_00 (36:35):
Aaron Powell You wrote a piece recently for, I
believe, the New York Times anddiscussing the AI race between
China and the US, and you putforward the idea, which I think
is a very good one, that Chinaand the US should sort of agree
to a strategic arms limitationtreaty.
(36:56):
Can you explain that to ourlisteners, wh why you think
that's important?
SPEAKER_02 (37:01):
When we imagine AI in the hands of bad people doing
bad things to us, there are twocategories of threat.
One is that an enemy countrylike China might get powerful AI
and do something bad to us.
But we know from the Cold Warthat the central way of dealing
with that type of, you know,peer competitor threat with
nuclear weapons was you bothrace and you achieve a rough
(37:24):
parity, and then there ismutually assured destruction.
And yes, on top, you can try tocontain the cost of the arms
race by having caps on thenumber of warheads each side
deploys.
But the fundamental deliverer ofpeace is mutually assured
destruction, the balance ofpower.
Meanwhile, there is a separatecategory of threat, which is
(37:44):
that bad actors, they could becriminals, they could be
terrorists, they could be NorthKorea, various kinds of rogue
get this technology and startusing it to hack everybody's
bank accounts or do a bioweaponor flood a city by making a damn
malfunction.
And that category can't bedeterred by mutually assured
destruction.
(38:04):
It's not a bilateral thing witha balance, it's a multipolar
thing with lots of rogues whohave not much to lose.
And so for this, you need anon-proliferation architecture
where you desperately try tocontrol the way in which the
technology spreads around theworld.
And my argument is that withmythos now out, and the, you
(38:26):
know, it's it's not theoreticalanymore, we do have a model that
exists in the world that couldbe used to do grievous damage to
financial stability and so forthif it was in the hands of uh
criminals.
So we we really need theequivalent of a nuclear
non-proliferation treaty.
And the good news is that mostof the world would abide by a
set of rules aboutnon-proliferation if the US
(38:48):
demanded that.
Because most of the world istotally dependent on the US to
get any access to any AI.
Because we have the NVIDIAchips, the US has, you know, uh
all the big compute clusters,the best algorithms, as well as
a lot of other coercive power.
And if you're a French AI lablike Mistral or a Canadian one
(39:08):
like Cohere, you nonethelesswant to raise money in America,
sell to American customers, dealwith American chips, and so
forth.
So you're gonna do exactly whatyou're told by the US
government.
The only government that wouldignore US sort of
non-proliferation policy,potentially, is China.
So there does need to be a dealwith China.
You can't coerce them into this.
(39:29):
And so that's why I think it's ahigh priority right now for
statecraft to go to China andsay, look, you're an AI
superpower.
We are an AI superpower, neitherof us want this technology in
the hands of rogues andcriminals and terrorists.
You love regulating theinternet, Mr.
China.
So why wouldn't you want toregulate this too?
So let's do a deal where youstop just open sourcing or
(39:52):
putting putting out open weightsof these models, because you and
I both know that this means thatmythos type technology will be
in the hands of criminals byearly 2027.
None of us want that.
So let's talk about a deal.
SPEAKER_00 (40:04):
And you said in that piece that you had talked to
people in China who were open tothis idea of a non-proliferation
treaty.
Is that correct?
SPEAKER_02 (40:13):
Yeah, so so China does everything fast, and so
they published my book first,even though they had to
translate it and do a bunch ofother things to get it out.
But they were still first.
And I went out there for eightdays and went and met lots of AI
leaders, both in technologycompanies and in academia.
And I was surprised that many ofthem raised the question of AI
safety unprompted.
(40:34):
Um, and so I think that's a goodsign that at least at the elite
level, there is a conversationabout the need for more focus on
AI safety.
I'm not pretending that theCommunist Party, you know,
necessarily wants to stop racingagainst the US.
Indeed, you know, until veryrecently, the only sign of any
policy from the Trumpadministration was it wanted to
race against China.
So both sides were locked in arace, and it was rational for
(40:56):
each one to race if the otherone was racing.
Since Mythos, the USadministration has shown some
signs of rethinking.
And I think the Chinese onewould too, if, you know,
prompted by a delegation thatwent to China or some overture
that said, look, we both have ashared interest in in keeping
this out of the hands ofcriminals.
SPEAKER_00 (41:15):
So if we have the bad actor, let's say North
Korea, and they have developedtheir own version of Mythos or
something close to it, and theydecide to attack the Swift Money
Channel.
How is that blocked?
How how would it in the in thescheme of uh agreed-to
non-proliferation treaty, howwould the AI powers block uh
(41:40):
North Korea from disrupting thefinancial system?
SPEAKER_02 (41:43):
Uh it would be blocked because Korea would
would not be able to get themodel in the first place.
Um the deal would be that youknow, all powerful AI, which
traces back mostly to Americaand in some cases to China,
would only be disseminatedaround the world to countries
that agree to sign up for propersafety controls.
So if the Germans want to set uptheir own AI security institute,
(42:07):
which promises to monitor usagein their own country, then the
Germans can get sort of the theweights that can get the model
from Anthropic or GoogleDeepMind or whoever, and they
can parcel it out to usersinside Germany under some
licensing agreement where somemoney flows back to the creators
of the models.
(42:27):
But they would need to bepolicing it such that, you know,
if they saw somebody using itfor a criminal purpose, like
doing a cyber attack, they wouldshut the criminal down.
And so these national, it's abit like the IAEA monitors the
use of civilian nuclear power inorder to prevent the nuclear
material being repurposed forweapons.
But I think it would actually beeasier in this case, because you
(42:50):
know, to run these models, youneed these enormous data
centers.
And they're kind of easy tospot.
We know where they are, theydepend on American technology to
function.
And so we could police thosethings.
And we could say to Germany,look, if you want to carry on
using AI, you need to policeyour guys in your country.
And then North Korea presumablywould not agree to police
anything, and so they wouldn'tget the technology.
SPEAKER_00 (43:11):
Is there a uh AI policy emerging from the Trump
administration, or is it justthe answer to everything is
China, and therefore we have topress on?
SPEAKER_02 (43:20):
Aaron Powell Well, there was a hint um before the
Trump Xi summit from theadministration, I think it came
from Scott Besson, the TreasurySecretary, that AI would be on
the agenda for discussion inBeijing.
I'm actually not aware that itwas discussed.
I don't think there's beenpublic statements about it.
Maybe it was discussed behindclosed doors, or maybe actually
(43:40):
something as technical as thisis not best treated in a leader
to leader summit.
It's best done at a you knowtechnical negotiators in some
separate forum.
So I'd say that there was a hintof openness to this kind of
approach.
Furthermore, in terms ofdomestic regulation of AI
releases, the Trumpadministration.
Has done in 180.
(44:01):
I mean, it it began earliersaying, you know, we think that
any AI security institute is abit dubious, possibly it's woke
nonsense.
They sort of weren't interestedin maintaining the national AI
safety institute that had beencreated under the previous
administration.
They didn't absolutely abolishit, but they renamed it and
downgraded it.
And now all of a sudden, they'vewaded in and taken control of
(44:24):
the list of entities that'sallowed to get hold of the
anthropic mythos model.
Anthropic said at the beginning,we'll give this to about 40 or
so responsible entities that canuse mythos to harden their
systems against future cyberattacks.
And their intention at Anthropicwas to roll it out to other
users in quite rapid wavesthereafter to try and get as
(44:46):
many as possible players on theinternet to have the opportunity
to find bugs and vulnerabilitiesin their own system and to shore
them up.
But the Trump administration hasarrogated unto itself the power
to decide who gets it next.
And so far, they haven'tannounced anybody next.
So the whole thing is frozen.
So I don't think Anthropic'shappy with this because they
(45:08):
think that there's a non-policywhere there ought to be a
policy.
But in a way, it's animprovement over the previous
state of affairs where it wasjust open accelerationism, no
holes barred.
SPEAKER_00 (45:17):
What prevents a company like Anthropic from
assuming sort of overreachingregulation from the Trump
administration or from wherever?
What prevents them from saying,you know what, we're just going
to go to Toronto and set up shopthere and do our work there, and
we won't be harassed by Trumpadministration regulators or
(45:38):
Congress or whatever it mightbe?
And I presume that, you know,Prime Minister Prime Minister,
what's his name?
Carney's his name.
I assume that Prime MinisterKearney would more than welcome,
you know, the best AI mines inthe world to move to Toronto or
Montreal or whatever.
Is that something we're likelyto see if if regulation becomes
(45:59):
too onerous, or or is it justunrealistic because of, you
know, people live and have livesand data labs are in the US and
so on and so forth?
SPEAKER_02 (46:10):
Aaron Ross Powell Well, it's a bit like you know
saying when the US imposesfinancial sanctions on a
country, maybe the country couldjust use a bank that's located
in, you know, Russia orsomething.
But the problem is it doesn'twork if the bank is connected to
Swift and to the internationaldollar-based financial system.
And it's kind of the same withAI.
(46:30):
If Anthropic moved to Canada, itwould still be dependent on all
bits of the sort of technologysupply chain, which are American
controlled.
You've got NVIDIA chips, you'vegot, you know, memory chips that
might be from Micron, you know,based in Idaho, you've got all
kinds of cooling gear, which isprobably made in America, you've
got ASML, the Dutch firm thatmakes the lithography machines,
(46:51):
but is subject to American powerbecause it operates in America,
sells to America.
I mean, it's just too difficult.
You'd have to basically take theentirety of the supply chain,
lock stock and barrel, and moveit to Canada, and that's not
gonna happen.
SPEAKER_00 (47:05):
Yeah.
So um, last question, we'vetaken a lot of your time here,
but the last question is whatdoes the future look like to
you?
What what are we gonna see inthe next three to five years?
SPEAKER_02 (47:14):
Aaron Powell Well, I think the first thing to say is
that if you look back three tofive years, the progress has
been much, much faster than Ithink many people still realize.
You know, there's a slight uhsort of popular delusion, I
feel, sometimes, that, you know,oh yes, AI arrived in 2022 with
Chat GPT, and now we have AI.
Okay.
No, no, no.
That's not how you should thinkabout it.
You know, the first modelshallucinated abominably.
(47:36):
Within a few months they stoppedhallucinating.
Then there was systems that hadmuch longer memory, so you could
put a Tolstoy novel into thesystem and get it to summarize
it or comment on it.
Then you've got models thatcould deal with video and
pictures and take in verbalmessages, do audio.
Then you had ones that could domath and logic, they could
reason their way throughproblems.
(47:57):
Now you have the beginnings ofagentic systems that can do
multitask operations withoutbeing interrupted.
Some of them go on for as longas you know a few hours.
So the progress since 2022 hasbeen extraordinary.
And I think if anything, theprogress looking forward is
going to be even moreextraordinary.
Such that if you talk to people,say it anthropic, they believe
(48:18):
that by 2028, only a year and ahalf away, there is going to be
such progress that a model cancode up the next model.
You will have what's calledrecursive self-improvement.
And at this point, the cyclesbetween one model and the next
slightly better model shrinkbecause you no longer have slow
humans having to bang theirheads against the wall and come
(48:38):
up with some smart idea abouthow to improve things.
The code will do it by itself.
That's an absolutelyextraordinary notion.
And whether or not, you know,it's too simple, it's
exaggerated, the timeline isslightly longer, the point is,
you know, buckle up.
This is going to be very, veryfast progress.
I believe agentic systems arecoming, they're going to be more
powerful.
I think that's going to meanthreats in terms of biosecurity,
(49:00):
uh weapons being potentiallycreated by bad guys.
So the political debate aroundcontrolling these models and not
releasing them to the generalpublic is going to heat up the
popular backlash, which we seein the polls very strongly
already, about people justhating AI, hating the fact that
it is going to take their jobs,the data centers will push up
their electricity prices, uh,their kids will be corrupted
(49:24):
because they won't do theirhomework anymore, and they'll be
addicted to chatbots that kindof are worse than social media.
I mean, the long litany ofcomplaints about AI, which is
already, I think it's thefastest growing popular backlash
against just about anything.
I was reading a piece todayabout this.
And and it might accelerate.
So wow, what does that mean?
So I think if you think about,you know, the way the Industrial
(49:45):
Revolution led to the ideasabout class struggle,
revolution, Marxism.
And and this revolution may takeplace ten times as fast and even
be ten times bigger.
Really buckle up.
SPEAKER_00 (49:59):
Yeah.
I mean, I think of it in termsof how many people really
understand the algorithms ofartificial intelligence.
I'd be surprised if the numberwas five million, right?
And you have eight billionpeople on the planet who don't
understand the algorithms of AI.
And so the power imbalance isunlike anything we've ever seen
(50:20):
in human history, right?
SPEAKER_01 (50:22):
Mm-hmm.
SPEAKER_00 (50:23):
So the eight billion probably not gonna feel
comfortable about the fivemillion, I would think.
SPEAKER_02 (50:30):
I think that's correct.
SPEAKER_00 (50:31):
Sebastian, thank you very much for your time.
We're uh right on the one hourmark, which we try to achieve
when we have great guests.
Thank you very much for doingthis.
SPEAKER_02 (50:39):
It's been a great pleasure, John.
Thank you.
Hello and welcome back to the News Items Podcast.
I'm John Ellis.
I'm the founder and editor oftwo Substack newsletters.
One is called News Items, theother is called Political News
Items, and you can find themboth at news-items.com.
Our guest today is SebastianMalaby.
Sebastian is the Paul A.
(00:20):
Volcker Senior Fellow forInternational Economics at the
Council on Foreign Relations, atwo-time Pulitzer Prize
finalist.
He's the author of six books,including More Money Than God,
Hedge Funds and the Making of aNew Elite, and The Power Law,
Venture Capital and the Makingof the New Future.
Sebastian, thank you very muchfor taking the time to be with
(00:42):
us today.
We're here to talk about yourlatest book about Demus
Hassabas, I think I have itpronounced right, finally.
SPEAKER_01 (00:49):
Right.
SPEAKER_00 (00:50):
And I think it's safe to say that Mr.
Hassabas is doing or is one ofthe very few people who is doing
uh the most important work onthe planet.
And before we get to the mostimportant work on the planet, I
thought it'd be helpful to thelisteners if you'd give us a
brief sketch of how Mr.
Hassabas got from being a childchess prodigy in his preteen
(01:13):
years to being the now chiefexecutive of Google Deep Mind, I
think the chief scientist andand the winner of the Nobel
Prize for Chemistry in 2024.
SPEAKER_02 (01:24):
Sure.
Well, Demis Hassabis was born toimmigrant parents in London.
His father was of Greek Cypriotorigin, his mother, Chinese
Singaporean, spent some time asan orphan on the streets of
Singapore as a child.
I like emphasizing this toAmerican and particularly
Silicon Valley audiences becauseit just gets across the point
that maybe Silicon Valley doesnot have a monopoly on melting
(01:47):
pot features or kind ofimmigrant founders and so forth.
Demis, you know, had this modestbackground, but his talents
proved to be immodest.
And when he saw his fatherplaying chess against his uncle,
aged about four, he just sort ofwatched the game a bit and then
he understood it, and then hecould beat adults quite soon
after that, and then he was thebest junior chess player in
(02:07):
Britain and second best in theworld, and pretty much expected
to become a chess professional.
But along the way, he startedwinning money from winning chess
tournaments, and he would buyhimself computers, which he
would upgrade when he got somemore money.
And he got into coding, and thesegue was actually reading a
book about chess programming sothat he could marry his interest
(02:28):
in computers with his interestin chess.
And the ideas in this chessprogramming manual kind of went
back to information theory, toTuring's early ideas, what is
computing?
And it, you know, struck him.
And a little bit later, he foundhimself, you know, he'd been
given a position to study atCambridge University when he was
extremely young.
And the university told him,listen, you're academically
(02:51):
ready, but you shouldn't comeyet because you're not socially
ready.
I think he was about 16 or so.
Um so go do something else for abit.
So he went off and worked for avideo game design company using
his coding skills.
And in this community of kind ofslightly autodidact misfits who
were building video gamestogether, is this sort of early
1990s?
He would be discussing how youcould make these games more
(03:14):
enthralling and interesting.
And that included discussions ofartificial intelligence.
The idea that, you know, theplayer of the game could treat
characters in the game incertain ways that would
encourage them to do more ofsomething or less of something
else.
This would be a basicreinforcement model, as people
calling it now in AI.
So ideas about AI started topercolate when he was around 17.
(03:38):
He read this book,Gürtel-Escherbach, which,
amongst many things it says,posits the notion that, you
know, basically the human brainruns on ones and zeros.
And so one day a computer with alot of ones and zeros and a big
enough architecture could dowhatever the human brain could
do.
And so putting this together, bythe time he was going to
Cambridge at the age of around18, Demis had formed this
(04:02):
ambition to create superpowerfulAI, which is just an astonishing
fact in itself.
I mean, this is somebody thatyoung and having this view in
1994, when we are kind of 18years before AI being able to do
a single thing.
I mean, it couldn't recognizethe photograph of a cat until
2012.
(04:22):
And so this early conviction ispart of what makes Demian such a
fascinating character.
SPEAKER_00 (04:27):
How did you come to get the access that you did?
I believe you had 30 interviewswith Mr.
Hosabas over uh you know anumber of years.
Did he approach you?
Did you approach him?
How did it work?
SPEAKER_02 (04:40):
Aaron Ross Powell I approached him.
Um I had met him um at techconferences because my previous
book was about uh venturecapital investing, and so I was
interested in technology.
I would go to tech conferencesin Europe and there would be
this sort of unlikely figure whowould show up looking about 25
years old, although he wasprobably more like 35 or 40 at
(05:01):
the time.
And uh and he would, you know,have this approachable, boyish
demeanor, extremelyunpretentious, kind of like the
kid next door who says, Hey,let's get a sandwich and uh
we'll do the dishes after lunchand then go for a walk.
How does that sound?
I mean, just super, superrelatable.
And then he would get on thestage and start talking, and
ideas about neuroscience andcomputer science and biology and
(05:24):
chemistry and physics and thehistory of movies and philosophy
would just tumble out of hismouth in this astonishing
stream.
So the juxtaposition between theapproachability of the manner
and the ambition of the ideasalways grabbed me.
And then around about 2022, whenI was thinking about the next
project after writing aboutventure capital, it struck me
(05:45):
that AI was sort of ripe.
I mean, ChatGPT was not out yet,but you'd seen examples of
powerful AI in the protein shapeprediction project that Demis
Susabus had done with GoogleDeepMind.
So I felt that the technologywas ripe, and this was an
amazing character.
And if I could get him to agreeto talk to me, I would be able
(06:06):
to tell the story of the makingof modern AI through this
unbelievable personality.
So I went to pitch him, and thepitch was basically: look, I've
listened to all your lectures,Demis.
And in these lectures, you tellus that the invention of
artificial intelligence is goingto be the most consequential
invention of human history.
Now, if that's true, then itfollows that you're going to be
(06:26):
one of the most important peoplein human history.
So, Demis, you don't have achoice.
There will be books about you.
Let's just get real here.
Don't pretend that you can hide.
And furthermore, you should wantthere to be books because you
cannot unleash a technology thisdisruptive on the world, which
is going to make people raisetheir children differently, do
their jobs differently, conceiveof themselves as humans
(06:47):
differently, and not explain whyyou're doing it.
I mean, it's dangerous, it'sdisruptive.
You have to say what your valuesare, what your motives are, what
the point is.
And so you should welcomesomebody who wants, who comes in
the front door, not the backdoor, and you know, promises to
take your ideas seriously and todo a good, you know, careful,
multi-year project where you Ireally tell your story.
(07:09):
And he thought about that for abit and he accepted.
And then right around when heaccepted, AGPT came out.
So my fringe topic went to themainstream faster than I could
have imagined.
SPEAKER_00 (07:19):
Aaron Powell A famous historian once said it's
it's almost impossible to writea biography about someone you
don't like.
Only a good biography uh can bewritten by can be written about
someone that you do like.
Did you like him?
Do you like him?
SPEAKER_02 (07:34):
I do like him.
One of the interesting thingsactually is that even somebody
you massively respect and likeis going to have blind spots.
I mean, I think of this asrelationship advice as well as
just sort of biographer advice.
And with Demis, you know, sureenough, there were certain
things where he was not able tosee himself lucidly.
Uh so for example, he alwayssays he doesn't like to control
(07:57):
people, that his mother broughthim up with these Christian
values not to control people, torespect everybody.
And it's true that he he carrieshimself in a way that's you know
designed to be non-intimidating.
And I have funny stories about,you know, because I live in
London, Demis Osavis lives inLondon.
It turns out that I have afriend whose kids were at the
same school as Demis' kids, andthis friend of mine went over to
(08:20):
Demis's house to pick up his sonfrom a birthday party, and he
saw Demis and he recognized himfrom a YouTube video.
So he said, Oh, you're the AIguy.
And uh Demis said, Yeah, I'm theAI guy.
And my friend said to Demis, um,you know, I I I watched this
video about AI on YouTube, andwhat it strikes me is that you
could make a lot of progress ifyou had two AIs and they kind of
(08:42):
argued with each other and thesynthesis of their views might
be better than either one.
And so now you have the, youknow, totally non-technical
random friend of mine who becamea publisher after leaving
university lecturing the futureNobel laureate on what he should
do with his AI models.
So I said to my friend, look, sohow did Demis respond to your
useful advice on the future ofAI?
(09:02):
And my friend said, Well, Demissaid it was a very useful
insight.
So he is a nice guy, right?
Well, I know, I know in thiscompletely unplanned piece of
reporting that he is a reallynice guy.
But he says he doesn't controlpeople, which is ridiculous
because he runs a large companyand he's smarter than everybody
else and more articulate, and hewins 99% of the debates he has
(09:25):
with people.
So of course he controls them.
But anyway, he refused to seethat.
And we had an argument aboutthat.
SPEAKER_00 (09:30):
There are two sort of, it seems to me, two
milestone moments in the historyof Google.
One was the acquisition ofJeffrey Hinton's company, which
was really like a two-page memo.
But Mr.
Hinton is often called thegodfather of AI, and Google
acquired his brain, essentially,and he worked there for a number
(09:51):
of years.
And the second, obviously, is uhGoogle's acquisition of
DeepMind.
How did that come up that comeabout, I guess, is the question.
SPEAKER_02 (09:59):
Yeah, so Demis founded DeepMind in 2010.
And um, for the first three anda bit years, it was sort of
raising money from Peter Thielin Silicon Valley, raising money
from a few other investors.
Elon Musk put five milliondollars in, was a Singaporean,
uh Hong Kong-based investor aswell.
(10:21):
So there's, you know, it waskind of gathering momentum, but
it was a struggle to raiseenough money to afford enough
computing power to really take abig leap forward.
And then Google becameinterested in buying it, partly
because they had already boughtthe Jeff Hinton boutique you
mentioned.
And Jeff Hinton had recommendedbuying a Demis company because
he knew Demis, he'd met Demis,he knew Demis was extraordinary.
(10:43):
And so, based on thatrecommendation and on a model
that Demis' company DeepMind hadproduced after a couple of
years, which was the firstagentic games playing model.
It was a model that could justthrough trial and error, without
being given the rules,understand those Atari video
games from the 70s and 80s,games like Pong and Breakout and
(11:05):
Sequest and stuff like that.
So Owl was sufficientlyimpressed by the technological
progress and impressed byHinton's recommendation and
wanted to buy DeepMind.
And Demis, you know, was keenbecause he wanted to have
liberation from the hamsterwheel of fundraising from the
venture capitalists.
(11:25):
And in fact, Larry Page, um,they met at an Elon Musk
birthday party, but it's fun,these stories are fun because it
shows you how all of thesepeople were entangled, you know,
right at the start, you know, inthe 2010-2012 zone, the same
characters were circling aroundeach other, kind of friendly but
competitive, sometimes viciouslycompetitive, as we saw recently
(11:46):
in the in the Musk v.
Altman trial.
But anyway, um there was thisbirthday party of Elon Musk's,
and Larry Page suggested goingfor a walk with Demis.
And in this walk, he said toDemis, look, you have to decide.
You could probably build acompany as big as Google if you
want, but it will take you allyour energy.
If what you really are is ascientist, then you should come
(12:08):
and do your science with thebenefit of my resources and my
computing power.
And to Demis, that was ano-brainer.
He was much less interested inbecoming a multi-billionaire
than he was in being the personwho delivered powerful AI to the
world.
SPEAKER_00 (12:22):
Aaron Ross Powell And with the acquisition by
Google, he he becamesubstantially wealthy.
So he didn't even have to worryabout that anymore.
SPEAKER_02 (12:30):
Aaron Ross Powell Yes.
Um to any normal person, walkingoff with a hundred and something
million is extremely wealthy.
It just wasn't on the scale thatuh you know Larry Page composed.
SPEAKER_00 (12:40):
Aaron Powell But you get the sense from reading your
book that he doesn't he doesn'treally care.
I mean, obviously he caresbecause it makes it makes life
easier and he doesn't have tofly coach or whatever.
I mean, is it is is that right?
It does he really is that asecondary concern or even a
tertiary concern for him?
SPEAKER_02 (12:57):
Aaron Powell Yeah.
I mean, I he he would always sayto me he doesn't care about
money.
So one day I quizzed him, youknow, we would meet in this pub
near his home in North London,and at the back of the pub there
was a dusty staircase.
You'd go up there and there'd bea abandoned room which nobody
really used.
And so we'd get coffees and wesit up there for two hours at a
time.
(13:17):
And in one of these sessions Isaid, okay, you say you don't b
believe in you don't care aboutmoney, so let's just go down the
list here.
What kind of car does yourfamily drive?
And he said, Oh, I've forgottenwhat it's called, but I think
it's 11 years old or something.
Do you have a second home?
No.
Do you have a yacht?
Of course not.
Do you collect really expensiveart?
No.
Well, do you have any hobbies?
Yeah, I do actually have ahobby, it's true.
What's the hobby?
Well, I I really supportLiverpool Football Club.
(13:41):
And I did buy myself a couple ofseason tickets.
I said, How much does that cost?
And he said£3,000, which is like5,000 bucks.
And that's kind of the size ofit.
I mean, you know, perhaps thereare some things that he didn't
tell me.
I think I'd find out later thathe had done a fairly fancy
renovation to his home.
But still, it we're talking, youknow, a a very nice middle class
home, not a kind of crazymansion.
(14:03):
And uh yeah, so I don't think hecares about money.
SPEAKER_00 (14:05):
So it's all about the work.
SPEAKER_02 (14:06):
It's about science, it's about ego, it's about fame,
it's about a sort of messianicdesire to be the person who
brings artificial intelligenceinto the world.
And then beneath that, or behindthat, there is something else.
And this surprised me.
Uh there was one time we weremeeting, and instead of being
inside in the pub, it was such anice day, even, you know, even
in London, you get a few nicedays.
(14:28):
So we went out to a park and wesat in a cafe, and the people at
the next table were havingnormal cafe type conversations.
You know, my friend was sick,she went to hospital, oh, I hope
she's better, etc.
And I was sitting oppositeDemis, who was in this sort of
messianic riff.
And he started saying, at two inthe morning, Sebastian, when I'm
reading a scientific paper, Ifeel that reality is staring at
(14:50):
me in the face, calling to me,waiting to be figured out.
We have to figure it out.
It's absurd.
We don't know what gravity is,we don't know what time is.
There are so many mysteries inphysics which you haven't yet
figured out.
I mean, this table, Sebastian.
And then he banged his fist onthe table.
He said, It's made of atomsjumping around with spaces in
between.
Why is it solid?
We don't even understand that.
We need to understand thesethings, and that's why I'm
(15:11):
building AI, so that we cancrack these scientific mysteries
and understand the true fabricof reality.
And that is what I need to dobefore I die.
And if I can understand hownature functions, I will be
getting closer to theintelligent being that may
presumably have set upeverything in reality in a way
that is amenable for humans togo and do science and to
(15:32):
understand it.
I will get closer to thatintelligent design, which could
be my definition of God.
SPEAKER_00 (15:37):
That's the way I feel when I wake up in the
morning, you know.
Slightly different.
I think we have to talk aboutone of the major breakthroughs
that generated a ton ofheadlines, which was the
artificial intelligencedeveloped by Mr.
Hashabus that defeated theworld's greatest Go player.
(15:59):
Can you tell us that particularstory?
SPEAKER_02 (16:02):
Sure.
So after Google bought DeepMindin 2014, Demis went off to a
conference with the seniorleadership of Google.
So there he is with his newproprietors.
And one of them, Sergey Brynn,is a Go player.
And Demis, you know, drops acomment to the effect that
maybe, you know, an AI systemcould defeat a Go player.
(16:24):
And Sergei Brynn, who knewsomething about computing and
something about Go, said, no,no, that's ridiculous.
Surely that's impossible.
And that was the standard viewat the time, that the idea that
you could have an AI that waspowerful enough to solve the
incredibly intricate patternsthat you get in Go and to derive
meaning from them when thecombinatorial complexity of how
those, you know, all thepermutations of how those stones
(16:47):
could be placed on the board,they're just, you know, almost
infinitely large.
It's a 19 by 19 board.
So the very first move, there's361 possibilities, second move
360, next one 359, you multiplythat out, you very quickly get
to this insanely big number.
So Sergei Brynn was justchanneling the consensus when he
said, you can't possibly dothat.
And Demis said to himselfsilently, if he thinks it's
(17:10):
impossible, then it ought toimpress him if I do it.
And then, you know, we'll getall the funding we ever want.
So he went back to his friendDavid Silver, who was one of the
scientists at DeepMind and afriend of Demis's from
University of Cambridge whenthey both did computer science
together.
And he said, um, so David, youyou you did your PhD on um
primitive go-playing AI.
(17:32):
Do you think you could make oneactually that really worked?
And Dave Silver's view was yes,because there had been advances
between him finishing his PhDand the moment when Demis asked
him this question.
There had been adequate progressin both deep learning and
reinforcement learning, whichare the two main strands of
artificial intelligence, that ifyou combine the two with some
(17:53):
novel tactics, this ought to bea soluble problem.
So Demis said, great, we know,let's do it.
And um he gave him, gave DavidSilver the resources, backed
him, and it took about a coupleof years.
But in this famous game in thespring of 2016, Deep Mind sort
of staged a new version of thatdeep blue versus Gary Kasparov
(18:15):
moment for chess.
Right.
And this instead was Alpha Goversus Lisa Dole.
And Lisa Dole is a uh SouthKorean world champion.
And uh it was best of five.
There was a prize of a millionbucks for the winner, and um
AlphaGo won four games to oneagainst the human.
And to this day, when Demis goesto South Korea, he's mobbed by
(18:38):
people as the father of AlphaGo.
And I heard on his most recentvisit, you know, people rushed
up to him and demanded that hesign a copy of my book, which is
the moment I think when finallyDemis accepted that he was happy
that I wrote the book.
SPEAKER_00 (18:50):
So AlphaGo, if we I mean, we can there are a lot of
steps along the way, but if wego from Alpha Go to Alpha Fold,
can you inform the listeners ofwhat Alpha Fold is and why it's
so important?
SPEAKER_02 (19:05):
Aaron Powell Sure.
So after AlphaGo was finished,now we're in the spring of 2016,
and Demis is walking out of thesort of exhibition hall where
the game took place with hisfriend David Silver, and it
happens that behind them, asthey walk down the street, there
is a camera crew with a bigmicrophone, sort of one of those
kind of fuzzy wooden things thatthey stick out at you.
(19:27):
And it picks up the conversationbetween Demis and Dave.
And unbelievably, Demis issaying, you know, having just
collected the laurels of this gobreakthrough kind of 20 minutes
earlier, he's saying, right,Dave, now we can do protein
folding.
And so, you know, he rested onhis laurels for kind of 15
minutes and announced the nextchallenge.
(19:48):
And this next challenge issomething that went back to his
time at Cambridge, where he hada biologist friend who told him
about a Nobel laureate inbiology called Christian
Anfinson, who had delivered aNobel lecture in which he had
conjectured that if you take anamino acid strand and you look
at the DNA sequence on thestrand, the sequence is a sort
(20:10):
of code.
And if you could read the code,you would predict how that
strand of amino acid would folditself up into a beautiful,
intricate shape.
And this is the shape of theprotein that it forms.
So proteins are these sort ofmini building blocks in nature,
in our bodies, in plants,everywhere.
Um, and they're they're composedof these strands of amino acid
(20:32):
that kind of like aself-executing origami model
twist themselves into anintricate shape.
And so this was anextraordinarily ambitious idea,
right?
That you could predict a shapewith actually more permutations
in the way that that strand ofamino acid could fold itself up,
more permutations than even inGo.
But also is much moreconsequential than Go, because
if you could understand all theshapes of all the proteins in
(20:56):
nature, you could in turn designor choose molecules that would
bind onto these proteins if youwere trying to build a medicine,
or possibly if you were tryingto do some innovation in
material sciences and a bunch ofother applications.
And so solving this problem wassimilar to Go in the sense that
it was a big combinatorial spacewith almost infinity of
(21:21):
permutations.
And so that's why Demis, havingconquered Go, immediately
thought of protein folding next.
But it was even harder and moreconsequential.
So anyway, after that, Demis uhconvened a team to sort of focus
on this protein foldingchallenge.
It was a long and winding road.
This one took, yeah, just overfour years.
(21:41):
But at the end of 2020, he had asystem that could accurately
predict all these protein shapesjust by knowing the DNA
sequence.
And the practical consequencewas that it used to take a PhD
researcher about four or fiveyears sometimes to do the
intricate kind of X-raycrystallography, which is the
process of.
Physically peering at thisminute protein shape and mapping
(22:04):
it.
It's so difficult.
It would take four or fiveyears.
Now, a scientist can simply lookup the protein shape like a
Google search because all of theshapes have been discovered by
AlphaFold, Demis's program, andhe has open sourced them so
anybody anywhere in the worldwho is interested in structural
biology can look up all of theshapes in nature pretty much.
(22:24):
And this is the achievement forwhich Demis shared the Nobel
Prize in Chemistry in 2024.
SPEAKER_00 (22:30):
Did Alpha Fold then morph into isomorphic labs?
Is that what happened?
SPEAKER_02 (22:35):
Correct.
After open sourcing all theshapes of all the proteins and
kind of gifting that to science,the commercial side of Google
kicked in, maybe notunreasonably, and they decided
to persist in that line ofresearch, but in a company that
would be separate from DeepMind.
It's called Isomorphic Labs.
And they would keep theirdiscoveries proprietary.
(22:57):
So where we are today is thatwe've gone from AlphaFold 2,
which was the system thatunraveled all those protein
shapes in 2020.
And now we're on Alpha Fold 4,which is really sort of a suite
of AI products whichcollectively are designed to
bring AI into the drug discoveryprocess.
And I think also perhaps in notonly to the discovery process,
(23:20):
into the clinical trial processthat follows.
And the notion is that you couldspeed up the discovery and the
productization of new medicines,you know, maybe by an order of
magnitude.
Trevor Burrus, Jr.
SPEAKER_00 (23:32):
And this is, I mean, it's all part of the story where
you marry a new technology toscience.
So and the you know, in themapping of the human genome, you
you took uh the the basicscience of genomics and you
married it to supercomputing,and lo and behold, Craig uh
Venter gets us a rough draft ofthe human genome, what, I don't
(23:55):
know, ten or fifteen yearssooner than NIH was doing it by
hand, so to speak.
Here you're married now marryingcode with artificial
intelligence, and so leaps andbounds of discovery occur.
What what is happening in theisomorphic lab space?
Are they have they come out withuh a pharma product or or a new
(24:18):
way of looking at a medicalproblem?
Is there is there a commercialapplication yet?
SPEAKER_02 (24:22):
Well, again, it i the inventions at isomorphic are
proprietary.
They don't talk about them indetail.
But but I can give you a coupleof shed light a little bit.
I sat next to a dinner recently,the co-founder of BioNTech,
which was the Germanpharmaceutical company that,
partnering with Pfizer, producedthe first mRNA COVID vaccine.
(24:46):
What she told me, uh thisscientist, is that in the drug
discovery phase, there arecertain uh you know the the the
the process from kind of havingthe concept of a medicine that
you're looking for and actuallyfinding what seems to be a
plausible molecule that works.
In some cases that's reduced toone-tenth of the previous time,
is what she told me.
And of course, you know, thereare caveats, it depends slightly
(25:08):
on what type of medicine you'relooking for.
This protein folding technologyis more useful in some cases
than in others, but it really isa meaningful speed-up.
And I've met otherpharmaceutical researchers who
say, yes, you know, I use itevery week.
It's it's really, you know, it'sa great shortcut for research.
The challenge comes in gettingthrough the clinical trials,
(25:30):
because you know, so far it'sbeen impossible to persuade any
regulator to contemplateshrinking and the time that
takes.
There is a sort of starter idea,which to me makes perfect sense
of how you could shrink it,which is that in a clinical
trial, you have both a testgroup, which get the new therapy
that you've just developed, andthen you have a control group,
(25:51):
which is supposed to bedemographically as close as
possible to the test group, butthey just get the
state-of-the-art therapy,whatever that happens to be.
But of course, because it's thestate-of-art therapy, by
definition, that means thatmany, many, many patients in
advanced Western countries havereceived the therapy already.
And if you use AI to search outamongst those people who've
(26:12):
already been treated, a groupwhich has the same demographic
characteristics as the people inyour trial group, you're in
business.
You can just simulate what theoutcome is from the
state-of-the-art therapy withoutneeding to have a whole trial
for that.
And this would be one way ofshrinking the time it takes to
get through clinical trials.
And I suspect there are morethat could be as regulators
(26:33):
develop more confidence in theaccuracy of simulated AI testing
results, perhaps they will beless exigent in the kind of
actual life human testing that'srequired.
SPEAKER_00 (26:44):
Aaron Powell Aside from birthing isomorphic labs
and another company calledIneffable Intelligence.
What by the way, what isineffable ineffable
intelligence?
What does that company do?
SPEAKER_02 (26:56):
Aaron Ross Powell So that company is founded by David
Silver, the person I wasreferencing earlier, who uh was
the genius behind AlphaGo, uhalso did a system called
AlphaZero after that.
And his particular specialsource is that he is the master
of what's called reinforcementlearning.
And and and maybe to justexplain this for a second,
because I think it's superuseful to people who understand,
(27:19):
okay, I I I know there's thisthing called AI, but what is it
under the hood?
And I think just one level down,just to give people one more
step in understanding, there aretwo main types of artificial
intelligence.
There is deep learning, whereyou have just masses and masses
of data, and you have algorithmsthat can find patterns in the
data and induce predictions fromthat.
(27:40):
It's a bit like, you know, yousend a human being to the
library and you have them readall of the books in the library,
and they're smart enough thatthey remember all the books, and
they can start thinking acrossall this crystallized human
knowledge which they've read.
Then there's a different kind oflearning, reinforcement
learning, where you're in anenvironment, and for humans it's
the real world, and you take anaction, and then something
(28:02):
happens in response to youraction, and you learn through
trial and error how the worldworks.
So you know that if you pick aglass of water up, you can feel
the weight, because there'sgravity.
Okay, now you understandsomething.
You know that if you turn theglass upside down, the water
will splash on the floor.
All right, now you understandthe difference between a solid
and a liquid.
You know that if you drop theglass, it shatters on a concrete
floor, et cetera, et cetera.
(28:23):
So as you as you take actions inthe world, you learn from the
world.
And that is the notion behindthis other strand of AI called
reinforcement learning.
There were two sort of warringtribes in artificial
intelligence in the twent in the2000s.
Deep learning was, you know, theheadquarters was Toronto, Jeff
(28:43):
Hinton's lab.
And then in Edmonton, Alberta,also Canada, funnily enough,
there was another Brit.
Jeff Hinton is one Brit.
The other one was called RichSutton, and he was the guru of
reinforcement learning, and thatis where David Silver did his
PhD, learning from Rich Sutton.
So David Silver brings thisreinforcement learning approach
to DeepMind.
He has amazing successes inbuilding these game playing
(29:05):
agents.
And now what he's doing withIneffable is trying to take
agentic AI, reinforcementlearning-based AI, and push that
to the nth degree.
I mean, he's he's although Ihaven't talked to him since he
started the company, I talked tohim an awful lot from my book.
And I know what his vision waswhen I was talking to him at the
end of 2025 and why, to him,reinforcement learning was being
(29:29):
under-emphasized by the biglabs.
And he feels as though when youwhen you learn from data, as
deep learning does, bydefinition, you can only learn
from what's in the data.
The data is created by humans inone way or another.
And so that's a limiting factoron how clever AI can be.
If you want the AI to surpasshumans, to be really super
intelligent, it has to learn byessentially creating its own
(29:51):
data through trial and error.
So in Go or Chess, the best,best systems are ones that don't
study expert human games.
They learn purely by playinggames against each other and
learning through trial and errorwhich strategies in chess or Go
work better.
And David Silver wants to extendthat idea.
Like, for example, in medicine,you could start by training an
(30:12):
AI on all the medical journalarticles.
And in this way, it would inhalethe state-of-the-art human
understanding of medicine.
Or you could say, I know what,we've got these humans wandering
around with, you know, bootbands on their wrists or aura
rings on their fingers,collecting actual data on the
(30:33):
human.
And, you know, we could get someother data on the human in
various other ways.
And we know, you know, the heartrate, the sleep, and so forth
and so forth.
And now we're going to correlatethat with health outcomes.
And so we're just going tobypass what human doctors think
they know.
We're going to go directly tothe data, and agents are going
to find patterns.
(30:55):
They're going to sort of runlittle experiments and figure
out what really is drivinghealth outcomes.
Leapfrogging over crystallizedhuman wisdom, which may not be
as perfect as we think it is.
SPEAKER_00 (31:06):
Aaron Powell One of the things that's interesting to
me is in any situation where youhave uh an extraordinary talent,
it attracts other extraordinarytalent.
And Damas's company and hiscohort have produced any number
of extraordinary talentedpeople.
Who are the most remarkable, inyour view, that have sort of
(31:28):
spun out of DeepMind?
SPEAKER_02 (31:30):
I think David Silver is a standout.
And beyond that, there have beena number of others.
The founder, the CEO of Mistral,the French AI lab, Arthur
Mensch, is another alum ofDeepMind.
And there are probably someothers who I'm not thinking of,
but one of the characteristicsof this field is that you do
(31:52):
need a ton of money to train AIsystems.
So oftentimes when somebodyleaves DeepMind, they go work
for one of the other Frontierlabs as opposed to doing a
startup.
Although increasingly, if you'refamous enough, you can raise
crazy amounts of capital, andthat gets to a whole other
discussion about bubbles and soforth.
SPEAKER_00 (32:09):
Aaron Ross Powell Well that's the subject of your
previous book, so we'll we'llstick on this one.
We have to we started by sayingit's the most important work on
the planet, or at least, in inmy opinion, the most important
uh work on the planet.
What exactly is artificialgeneral intelligence?
SPEAKER_02 (32:26):
Aaron Ross Powell There are two debates in this
field which are impossible toresolve because people don't
agree on definitions.
One is the one you just asked,what is artificial general
intelligence?
The other one is can a computerbe conscious?
Because it depends on what yourdefinition of consciousness is.
But let me just answer yourquestion on artificial general
intelligence.
You could say, look, if you useGemini or Chat GPT or Claude
(32:48):
today, it's artificial, it'sgeneral, it can talk about a lot
of things, and it's intelligent.
We have it, right?
I mean, that would be aplausible definition, a literal
definition of AGI.
At the other extreme, you havesomeone like Demis who tends to
want to push the moment when wedeclare that we have AGI.
He wants to push that out intothe future because he's having
(33:08):
such a great time, you know,doing the research to try to get
to it.
He doesn't want the research tostop, but then he wants to
extend the journey.
And then maybe there's a littlebit also he doesn't want to
freak people out by saying, youknow, we now have this
scary-sounding thing.
So for whichever reason, helikes to say AGI is further in
the future.
And in order to back up hisfurther in the future point of
(33:30):
view, he chooses a definitionwhich is very stringent.
He says, okay, let's imagine wetrained an AI system on
everything that was known upuntil 1911, and then we just
waited and saw if the AI systemcame up with general relativity
all by itself.
It's a pretty tough hurdle.
SPEAKER_00 (33:47):
I would say so.
I wanted to ask you, and JeffHinton is coming out, I think
this fall, with a manifestoabout the dangers of AI.
How does how does Mr.
Hashabas, how does Demis viewthe perils of AI?
SPEAKER_02 (34:06):
Aaron Ross Powell I mean what he says is, and I kind
of agree with him, is that look,there's a non-zero risk, that's
his phrase, non-zero risk thatthese models will start to
attack humans.
And I actually used to thinkthis was a crazy notion, and I
thought, well, of course,machines would be more
intelligent than people.
We already see that in domainslike Go, and so why wouldn't it
(34:27):
happen more generally?
But machines wouldn't attackhumans because they're not
really motivated to do so,right?
We are evolved to want tosurvive, to pass on our DNA.
Machines don't have DNA, theydon't care about survival, so
therefore, why would they attackus?
That was my kind of comfortingworldview.
And then one day I went toToronto to see uh Jeff Hinton,
(34:48):
and I sat in his kitchen for acouple of hours, and I said,
Look, Jeff, you know, surelyyou're exaggerating.
Why would these systems attackus?
They don't want to survive.
And he said, Okay, we'll do thisthought experiment, Sebastian.
Imagine you have a very powerfulAI, and you're worried that an
enemy AI, maybe a Chinese AI ora Russian AI, is going to attack
your AI.
(35:09):
You're a slow-thinking human, soyou're way too dumb to respond
to a cyber attack at digitalspeed.
So you're going to tell your AI,listen, if you see the attack
coming, defend yourself.
Maybe counterattack.
Whatever you do, survive.
Survive.
We just used that word.
Now are you feeling comforted,Sebastian?
And I kind of see his point.
I mean, I think we will endowthese systems with a desire to
(35:33):
survive.
And indeed, that desire may sortof just emerge out of the
training data because thesesystems are trained on, you
know, novels and books about thehuman condition and all sorts of
stories about humans fightingfor survival.
And so why wouldn't they get theidea of survival?
And if you marry supremeintelligence with the urge to
(35:54):
survive, with an ability to beduplicitous, which is something
that we observe in the models,it strikes me that, yeah, they
could turn against us.
Now, of course, there's a wholebranch of AI research called
alignment research, which isprecisely devoted to avoiding
that outcome.
And, you know, very smart peopleare engaged in that.
So I hope that we can containthis risk.
(36:15):
I don't put this risk at even1%.
I think it's below that.
But I think non-zero is theright answer, and anybody who
claims that there's no risk orthat the risk is 50%, um, are
reflecting something about theirown psychology as opposed to a
sober reading of the evidence.
SPEAKER_00 (36:35):
Aaron Powell You wrote a piece recently for, I
believe, the New York Times anddiscussing the AI race between
China and the US, and you putforward the idea, which I think
is a very good one, that Chinaand the US should sort of agree
to a strategic arms limitationtreaty.
(36:56):
Can you explain that to ourlisteners, wh why you think
that's important?
SPEAKER_02 (37:01):
When we imagine AI in the hands of bad people doing
bad things to us, there are twocategories of threat.
One is that an enemy countrylike China might get powerful AI
and do something bad to us.
But we know from the Cold Warthat the central way of dealing
with that type of, you know,peer competitor threat with
nuclear weapons was you bothrace and you achieve a rough
(37:24):
parity, and then there ismutually assured destruction.
And yes, on top, you can try tocontain the cost of the arms
race by having caps on thenumber of warheads each side
deploys.
But the fundamental deliverer ofpeace is mutually assured
destruction, the balance ofpower.
Meanwhile, there is a separatecategory of threat, which is
(37:44):
that bad actors, they could becriminals, they could be
terrorists, they could be NorthKorea, various kinds of rogue
get this technology and startusing it to hack everybody's
bank accounts or do a bioweaponor flood a city by making a damn
malfunction.
And that category can't bedeterred by mutually assured
destruction.
(38:04):
It's not a bilateral thing witha balance, it's a multipolar
thing with lots of rogues whohave not much to lose.
And so for this, you need anon-proliferation architecture
where you desperately try tocontrol the way in which the
technology spreads around theworld.
And my argument is that withmythos now out, and the, you
(38:26):
know, it's it's not theoreticalanymore, we do have a model that
exists in the world that couldbe used to do grievous damage to
financial stability and so forthif it was in the hands of uh
criminals.
So we we really need theequivalent of a nuclear
non-proliferation treaty.
And the good news is that mostof the world would abide by a
set of rules aboutnon-proliferation if the US
(38:48):
demanded that.
Because most of the world istotally dependent on the US to
get any access to any AI.
Because we have the NVIDIAchips, the US has, you know, uh
all the big compute clusters,the best algorithms, as well as
a lot of other coercive power.
And if you're a French AI lablike Mistral or a Canadian one
(39:08):
like Cohere, you nonethelesswant to raise money in America,
sell to American customers, dealwith American chips, and so
forth.
So you're gonna do exactly whatyou're told by the US
government.
The only government that wouldignore US sort of
non-proliferation policy,potentially, is China.
So there does need to be a dealwith China.
You can't coerce them into this.
(39:29):
And so that's why I think it's ahigh priority right now for
statecraft to go to China andsay, look, you're an AI
superpower.
We are an AI superpower, neitherof us want this technology in
the hands of rogues andcriminals and terrorists.
You love regulating theinternet, Mr.
China.
So why wouldn't you want toregulate this too?
So let's do a deal where youstop just open sourcing or
(39:52):
putting putting out open weightsof these models, because you and
I both know that this means thatmythos type technology will be
in the hands of criminals byearly 2027.
None of us want that.
So let's talk about a deal.
SPEAKER_00 (40:04):
And you said in that piece that you had talked to
people in China who were open tothis idea of a non-proliferation
treaty.
Is that correct?
SPEAKER_02 (40:13):
Yeah, so so China does everything fast, and so
they published my book first,even though they had to
translate it and do a bunch ofother things to get it out.
But they were still first.
And I went out there for eightdays and went and met lots of AI
leaders, both in technologycompanies and in academia.
And I was surprised that many ofthem raised the question of AI
safety unprompted.
(40:34):
Um, and so I think that's a goodsign that at least at the elite
level, there is a conversationabout the need for more focus on
AI safety.
I'm not pretending that theCommunist Party, you know,
necessarily wants to stop racingagainst the US.
Indeed, you know, until veryrecently, the only sign of any
policy from the Trumpadministration was it wanted to
race against China.
So both sides were locked in arace, and it was rational for
(40:56):
each one to race if the otherone was racing.
Since Mythos, the USadministration has shown some
signs of rethinking.
And I think the Chinese onewould too, if, you know,
prompted by a delegation thatwent to China or some overture
that said, look, we both have ashared interest in in keeping
this out of the hands ofcriminals.
SPEAKER_00 (41:15):
So if we have the bad actor, let's say North
Korea, and they have developedtheir own version of Mythos or
something close to it, and theydecide to attack the Swift Money
Channel.
How is that blocked?
How how would it in the in thescheme of uh agreed-to
non-proliferation treaty, howwould the AI powers block uh
(41:40):
North Korea from disrupting thefinancial system?
SPEAKER_02 (41:43):
Uh it would be blocked because Korea would
would not be able to get themodel in the first place.
Um the deal would be that youknow, all powerful AI, which
traces back mostly to Americaand in some cases to China,
would only be disseminatedaround the world to countries
that agree to sign up for propersafety controls.
So if the Germans want to set uptheir own AI security institute,
(42:07):
which promises to monitor usagein their own country, then the
Germans can get sort of the theweights that can get the model
from Anthropic or GoogleDeepMind or whoever, and they
can parcel it out to usersinside Germany under some
licensing agreement where somemoney flows back to the creators
of the models.
(42:27):
But they would need to bepolicing it such that, you know,
if they saw somebody using itfor a criminal purpose, like
doing a cyber attack, they wouldshut the criminal down.
And so these national, it's abit like the IAEA monitors the
use of civilian nuclear power inorder to prevent the nuclear
material being repurposed forweapons.
But I think it would actually beeasier in this case, because you
(42:50):
know, to run these models, youneed these enormous data
centers.
And they're kind of easy tospot.
We know where they are, theydepend on American technology to
function.
And so we could police thosethings.
And we could say to Germany,look, if you want to carry on
using AI, you need to policeyour guys in your country.
And then North Korea presumablywould not agree to police
anything, and so they wouldn'tget the technology.
SPEAKER_00 (43:11):
Is there a uh AI policy emerging from the Trump
administration, or is it justthe answer to everything is
China, and therefore we have topress on?
SPEAKER_02 (43:20):
Aaron Powell Well, there was a hint um before the
Trump Xi summit from theadministration, I think it came
from Scott Besson, the TreasurySecretary, that AI would be on
the agenda for discussion inBeijing.
I'm actually not aware that itwas discussed.
I don't think there's beenpublic statements about it.
Maybe it was discussed behindclosed doors, or maybe actually
(43:40):
something as technical as thisis not best treated in a leader
to leader summit.
It's best done at a you knowtechnical negotiators in some
separate forum.
So I'd say that there was a hintof openness to this kind of
approach.
Furthermore, in terms ofdomestic regulation of AI
releases, the Trumpadministration.
Has done in 180.
(44:01):
I mean, it it began earliersaying, you know, we think that
any AI security institute is abit dubious, possibly it's woke
nonsense.
They sort of weren't interestedin maintaining the national AI
safety institute that had beencreated under the previous
administration.
They didn't absolutely abolishit, but they renamed it and
downgraded it.
And now all of a sudden, they'vewaded in and taken control of
(44:24):
the list of entities that'sallowed to get hold of the
anthropic mythos model.
Anthropic said at the beginning,we'll give this to about 40 or
so responsible entities that canuse mythos to harden their
systems against future cyberattacks.
And their intention at Anthropicwas to roll it out to other
users in quite rapid wavesthereafter to try and get as
(44:46):
many as possible players on theinternet to have the opportunity
to find bugs and vulnerabilitiesin their own system and to shore
them up.
But the Trump administration hasarrogated unto itself the power
to decide who gets it next.
And so far, they haven'tannounced anybody next.
So the whole thing is frozen.
So I don't think Anthropic'shappy with this because they
(45:08):
think that there's a non-policywhere there ought to be a
policy.
But in a way, it's animprovement over the previous
state of affairs where it wasjust open accelerationism, no
holes barred.
SPEAKER_00 (45:17):
What prevents a company like Anthropic from
assuming sort of overreachingregulation from the Trump
administration or from wherever?
What prevents them from saying,you know what, we're just going
to go to Toronto and set up shopthere and do our work there, and
we won't be harassed by Trumpadministration regulators or
(45:38):
Congress or whatever it mightbe?
And I presume that, you know,Prime Minister Prime Minister,
what's his name?
Carney's his name.
I assume that Prime MinisterKearney would more than welcome,
you know, the best AI mines inthe world to move to Toronto or
Montreal or whatever.
Is that something we're likelyto see if if regulation becomes
(45:59):
too onerous, or or is it justunrealistic because of, you
know, people live and have livesand data labs are in the US and
so on and so forth?
SPEAKER_02 (46:10):
Aaron Ross Powell Well, it's a bit like you know
saying when the US imposesfinancial sanctions on a
country, maybe the country couldjust use a bank that's located
in, you know, Russia orsomething.
But the problem is it doesn'twork if the bank is connected to
Swift and to the internationaldollar-based financial system.
And it's kind of the same withAI.
(46:30):
If Anthropic moved to Canada, itwould still be dependent on all
bits of the sort of technologysupply chain, which are American
controlled.
You've got NVIDIA chips, you'vegot, you know, memory chips that
might be from Micron, you know,based in Idaho, you've got all
kinds of cooling gear, which isprobably made in America, you've
got ASML, the Dutch firm thatmakes the lithography machines,
(46:51):
but is subject to American powerbecause it operates in America,
sells to America.
I mean, it's just too difficult.
You'd have to basically take theentirety of the supply chain,
lock stock and barrel, and moveit to Canada, and that's not
gonna happen.
SPEAKER_00 (47:05):
Yeah.
So um, last question, we'vetaken a lot of your time here,
but the last question is whatdoes the future look like to
you?
What what are we gonna see inthe next three to five years?
SPEAKER_02 (47:14):
Aaron Powell Well, I think the first thing to say is
that if you look back three tofive years, the progress has
been much, much faster than Ithink many people still realize.
You know, there's a slight uhsort of popular delusion, I
feel, sometimes, that, you know,oh yes, AI arrived in 2022 with
Chat GPT, and now we have AI.
Okay.
No, no, no.
That's not how you should thinkabout it.
You know, the first modelshallucinated abominably.
(47:36):
Within a few months they stoppedhallucinating.
Then there was systems that hadmuch longer memory, so you could
put a Tolstoy novel into thesystem and get it to summarize
it or comment on it.
Then you've got models thatcould deal with video and
pictures and take in verbalmessages, do audio.
Then you had ones that could domath and logic, they could
reason their way throughproblems.
(47:57):
Now you have the beginnings ofagentic systems that can do
multitask operations withoutbeing interrupted.
Some of them go on for as longas you know a few hours.
So the progress since 2022 hasbeen extraordinary.
And I think if anything, theprogress looking forward is
going to be even moreextraordinary.
Such that if you talk to people,say it anthropic, they believe
(48:18):
that by 2028, only a year and ahalf away, there is going to be
such progress that a model cancode up the next model.
You will have what's calledrecursive self-improvement.
And at this point, the cyclesbetween one model and the next
slightly better model shrinkbecause you no longer have slow
humans having to bang theirheads against the wall and come
(48:38):
up with some smart idea abouthow to improve things.
The code will do it by itself.
That's an absolutelyextraordinary notion.
And whether or not, you know,it's too simple, it's
exaggerated, the timeline isslightly longer, the point is,
you know, buckle up.
This is going to be very, veryfast progress.
I believe agentic systems arecoming, they're going to be more
powerful.
I think that's going to meanthreats in terms of biosecurity,
(49:00):
uh weapons being potentiallycreated by bad guys.
So the political debate aroundcontrolling these models and not
releasing them to the generalpublic is going to heat up the
popular backlash, which we seein the polls very strongly
already, about people justhating AI, hating the fact that
it is going to take their jobs,the data centers will push up
their electricity prices, uh,their kids will be corrupted
(49:24):
because they won't do theirhomework anymore, and they'll be
addicted to chatbots that kindof are worse than social media.
I mean, the long litany ofcomplaints about AI, which is
already, I think it's thefastest growing popular backlash
against just about anything.
I was reading a piece todayabout this.
And and it might accelerate.
So wow, what does that mean?
So I think if you think about,you know, the way the Industrial
(49:45):
Revolution led to the ideasabout class struggle,
revolution, Marxism.
And and this revolution may takeplace ten times as fast and even
be ten times bigger.
Really buckle up.
SPEAKER_00 (49:59):
Yeah.
I mean, I think of it in termsof how many people really
understand the algorithms ofartificial intelligence.
I'd be surprised if the numberwas five million, right?
And you have eight billionpeople on the planet who don't
understand the algorithms of AI.
And so the power imbalance isunlike anything we've ever seen
(50:20):
in human history, right?
SPEAKER_01 (50:22):
Mm-hmm.
SPEAKER_00 (50:23):
So the eight billion probably not gonna feel
comfortable about the fivemillion, I would think.
SPEAKER_02 (50:30):
I think that's correct.
SPEAKER_00 (50:31):
Sebastian, thank you very much for your time.
We're uh right on the one hourmark, which we try to achieve
when we have great guests.
Thank you very much for doingthis.
SPEAKER_02 (50:39):
It's been a great pleasure, John.
Thank you.
Popular Podcasts
Betrayal Weekly
Betrayal Weekly is back for a new season. Every Thursday, Betrayal Weekly shares first-hand accounts of broken trust, shocking deceptions, and the trail of destruction they leave behind. Hosted by Andrea Gunning, this weekly ongoing series digs into real-life stories of betrayal and the aftermath. From stories of double lives to dark discoveries, these are cautionary tales and accounts of resilience against all odds. From the producers of the critically acclaimed Betrayal series, Betrayal Weekly drops new episodes every Thursday. If you would like to share your story, you can reach out to the Betrayal Team by emailing them at betrayalpod@gmail.com and follow us on Instagram at @betrayalpod and @glasspodcasts. Please join our Substack for additional exclusive content, curated book recommendations, and community discussions. Sign up FREE by clicking this link Beyond Betrayal Substack. Join our community dedicated to truth, resilience, and healing. Your voice matters! Be a part of our Betrayal journey on Substack.
Stuff You Should Know
If you've ever wanted to know about champagne, satanism, the Stonewall Uprising, chaos theory, LSD, El Nino, true crime and Rosa Parks, then look no further. Josh and Chuck have you covered.
Dateline NBC
Current and classic episodes, featuring compelling true-crime mysteries, powerful documentaries and in-depth investigations. Follow now to get the latest episodes of Dateline NBC completely free, or subscribe to Dateline Premium for ad-free listening and exclusive bonus content: DatelinePremium.com