May 2, 2026 • 15 mins
No-one knows what the future of AI has in store for us, and one journalist has raised concerns about the impact of the technology.
When investigative journalist Karen Hao started looking into Sam Altman’s OpenAI, she had hopes for the technology, but extensive research and unparalleled access to those closest to the AI arms race left her with a different view.
Her work in this space has made her one of the foremost tech journos covering AI. She’s been listed in Time Magazine’s 100 Most Influential People.
She's heading to New Zealand for the Auckland Writers Festival with her book EMPIRE OF AI: Inside the Reckless Race for Total Domination.
"My criticism of companies that use this kind of rhetoric is that they are essentially just leveraging the lack of a shared definition as a way to just hype up their technologies."
LISTEN ABOVE
See omnystudio.com/listener for privacy information.
Listen
Watch
Mark as Played
Transcript
Episode Transcript
Available transcripts are automatically generated. Complete accuracy is not guaranteed.
Speaker 1 (00:06):
You're listening to the Sunday Session podcast with Francesca Rudkin
from News TALKSB So.
Speaker 2 (00:13):
Just what does the future of AI have in store
for us all? None of us really know, do we?
Seven years ago, when investigative journalist Karen Howe started looking
into Sam Oltman's Open AI, she thought they were one
of the good guys. Extensive research and unparalleled access to
those closest to the AI race have left Karen with
a much different view of tech companies and those leading
(00:33):
the charge who work in this space has made her
one of the most foremost tech journals covering AI. She's
been listed in Time Magazine's one hundred Most Influential People,
and Karen is heading to New Zealand for the Auckland
Writers Festival with her book Empire of AI and Karen
Howe's with me now.
Speaker 3 (00:49):
Good morning, Hi Francesca, thank you so much for having me.
Speaker 2 (00:53):
Firstly, and I'm sure my audience knows this, but can
we just define the term AI and AGI?
Speaker 3 (01:00):
Yes? So, I always like to say that the word
AI is like the word transportation. It refers to such
a wide collection of technologies that can be as different
as different from a bicycle to a rocket, and really
these technologies are just loosely tied together based on the
fact that they are each trying to mimic a certain
(01:22):
aspect of human capability. So there are systems that mimic
our language abilities, there are systems that mimic our listening abilities,
are visual recognition and it Ultimately each of these these
different types of AI technologies can actually be vastly different
(01:44):
and have very different cost benefit trade offs. AGI is
a term that refers to artificial general intelligence and is
meant to evoke this idea that somehow the ultimate quest
for building AI technologies is to replicate all human capabilities
(02:05):
in a single system. But one of the problems with
AGI is that it's very poorly defined, in that we
don't really have good ways of defining what human intelligence
is and whether or not we've actually reached the goalposts
of AGI. And so my criticism of companies that use
this kind of rhetoric is that they are essentially just
(02:27):
leveraging the lack of a shared definition as a way
to just hype up their technologies.
Speaker 2 (02:33):
Bringant, thank you for starting us off with it. That's excellent.
Is AGI about science? The betament of humankind, power or
money or all of the above.
Speaker 3 (02:44):
Primarily it's about power and money. AGI at this point
is you could say, almost a quasi religious quest, and
just like other religions, there is this idea that it
is ultimately meant to bring people who believe in to heaven,
(03:07):
and it could also be a tool that damns people
to hell. But the reason why this is such a
potent narrative and is used so often is because companies
that are saying that they're building AGI are ultimately able
to use this to justify the accruval of an enormous
amount of money and power. So all of the tech
(03:29):
companies that are developing AI and using the AGI narrative,
like open Ai, Anthropic, Xai, Google, they now are the
most valuable companies in the world, and they still want
to continue accruing yet more capital to build their technologies.
(03:51):
They are deploying more capital than ever any other industry
ever before to build massive infrastructure projects all around the world. So, yeah,
at the heart of this conversation is indeed money and power,
and we'll.
Speaker 2 (04:08):
Dig into that shortly. But AI open AI is sort
of the center of the book that you've written, and
open AI began as an ultruistic organization. It was a
nonprofit open organization, is implied by the title, It had
this idealistic governance structure. What was its original objective?
Speaker 3 (04:28):
So I have a different answer for this now versus
when I first started covering Open AI. Back then, I
thought they had this kind of pure hearted, mission driven
objective of trying to ensure that AI advancement ultimately goes
well and benefits all of humanity, and that then there
was some kind of profit motive that slowly corrupted the
(04:52):
organization over time. But in hindsight, I think they're real objective.
Their objective that wasn't actually explicitly stated when public messaging
was a desire to become the dominant AI lab in
the world or the dominant force shaping AI technology development
(05:14):
in the world, and you could say that that has
never changed, and what we're seeing now is just a
continued manifestation of this desire for dominance.
Speaker 2 (05:27):
Because the hugely competitive these II companies. This is what
you learn through the book. There is this drive to
be first, these ideological clashes, which makes each player think
that they need to be first to deliver AI for
their purpose because it's the right purpose, and yet that
ambition doesn't quite sit with the ethically minded AI that
they claim to sort of start out with, does it.
Speaker 3 (05:51):
That's exactly right. I mean, the thing that's so interesting
is I often use this analogy that the AI world
is a little bit like Doune The Science Fiction Network
by Frank Herbert. And the reason is because in doing
the main character poly treatise, he knows that there are
(06:11):
these myths that are seated on different planets, including his
own oracus, and that that myth help portrays himself as
the coming of the Messiah, and that if he steps
into the myth, he can leverage it to control the people.
And so he steps into it, but he genuinely believes
that this is to ultimately achieve a good purpose, like
(06:34):
he is doing it in a morally righteous way. And
then he also begins to lose himself in the myth
because every day he's living and embodying this myth, and
the myth and reality start to blur for him, and
he starts to wonder whether or not actually, in fact,
he is the Messiah and the myth is real. And
this is just such an applicable metaphor for how the
(06:58):
AI world works because there's all this myth making about
AGI that many people know is myth making. They are
leveraging it as this marketing rhetoric to accurrue more money
and power to the company. But they also exist in
this space where myth and reality are blurred. They breathe
(07:18):
this myth day in and day out, and they begin
to wonder whether or not it's true, And at the
end of the day they're like, irrespective of whether or
not it's true, both myth making and this quest is
ultimately for good. We are in the moral we are
we are doing something morally righteous. And so there is
(07:39):
this degree of like to say it in a in
like a not too flattering way, self delusion that happens
within the AI world, where people are the revealed pursuit
is of money and power, but their stated pursuit continues
to be this really these like high minded ideals where
(08:02):
if you talk with most people in the AIR world,
they genuinely believe that they are on the quest to
bringing benefit to all of humanity.
Speaker 2 (08:10):
And Karen, who is shaping AI is important, isn't it?
Is it being shaped by too small a pull of people?
Speaker 3 (08:18):
Absolutely? I mean, I think that is the central problem
of the A industry today. And the reason why I
call these companies empires of AI is because they essentially
are these entities that have accrued an enormous amount of power,
and ultimately they are these small number of people are
(08:42):
dictating to billions of people around the world how this
technology that is fundamentally consequential to the future is going
to go, and therefore how everyone in the future is
meant to live and relate to each other and to
work without any input from those billions of people. And
(09:03):
so it doesn't really matter like who is the top
in this kind of scenario, as long as there's this
power structure that enables a small group of people to
dictate these things to billions of people without any accountability,
it really doesn't like just switching out the cast of
characters at the top doesn't really solve the problem.
Speaker 2 (09:26):
It's a very broad question, and I know that it
can't be a quick answer, but you know, maybe you
could just touch on the social and environmental costs of AGI.
Speaker 3 (09:36):
So I don't think people fully understand that AI as
a digital technology has actually a very physical and involved
supply chain that also includes an enormous amount of human labor.
So there are two aspects of AI development that are
(09:58):
often overlooked because of this lack of awareness about that
supply chain. One is that there's an enormous amount of
labor exploitation that happens as part of the supply chain,
and the other is that there's this massive physical manifestation
of AI development, which comes in the form of data
centers and supercomputers that are popping up all around the world.
(10:21):
When it comes to the labor exploitation, I talk in
my book about these content moderators that Opening Eye contracted
in Kenya to build a content moderation filter that they
ultimately put on chat GBT to ensure that users would
never be exposed to the horrific, toxic content that they
(10:44):
were scraping from Reddit and other social media forums to
train their AI models. And in the same way as
content moderators in social media are these content moderators, their
psychological state of mind just completely devolved. And not only
did they break down as individuals, their families broke down,
(11:04):
their communities broke down because they are part of a
fabric in society and when they can't support themselves, there's
suddenly a hole in that fabric, and that fabric begins
to unwind the other aspect, the environmental aspect. People. You know,
people are familiar with data centers. Generally speaking, people understand
(11:26):
data centers to be essential to our modern digitally mediated lives.
But what is not well understood is the degree to
which AI data centers are of a fundamentally different breed
of data center. So Meta's first data center in twenty eleven,
compared to their current AI supercomputer that they are building
(11:46):
in Louisiana, the supercomputer is four hundred times the size
of that first data center, and it is on track
to be one fifth the size of Manhattan and to
eventually use the same amount of power as the average
power demand in all of New York City. So these
(12:07):
facilities are the size of cities and use the power
demand of cities, and there are like they're popping up
all around the world, Like this is just one facility
that I'm talking about, And so the data center expansion
is single handedly beginning to reverse the climate gains that
(12:32):
we made in the last decade, because what are these
facilities actually being powered by. They're being powered by fossil fuels.
The technistry loves to talk this big game about how
they're going to use nuclear to power these facilities with
more clean with clean energy, But the nuclear build out
(12:55):
would not be able to happen nearly fast enough for
supporting this data center build out. And so what we're
actually seeing is coal plants that were meant to be
retired that are instead having their lives extended, natural gas plants,
new natural gas plants that are being built, and there
(13:17):
have been many many cases in which these plants or
these plants end up. You know, they're not just spewing carbon,
they're also spewing air pollutants into communities. And there's an
infamous case of Elon Musk building a giant supercomputer called
(13:37):
Colossus in Memphis, Tennessee, and using thirty five methane gas
turbines to power this facility, and the community that exists
near this facility found out that these methane gas turbines
had popped up overnight when they started smelling what smelled
like a gas leak in their homes, and they suddenly
discovered that they know they were having their right to
(14:02):
clean air taken away from them. So there's just all
of these inner law sucking harms that are very hidden
to the average consumer of AI technologies. When it comes
to this technology's development.
Speaker 2 (14:18):
Karen, AI is moving, it's you predict how it's going
to continue to develop.
Speaker 3 (14:22):
I don't engage in predictions because predictions assume that the
future's already written, and all of my work is ultimately
about how we today, based on our actions, create the
future together. And so what I hope will happen with
the AI industry and with AI development is that everyone
(14:44):
listening to this program, everyone that's thinking about AI and
thinking about how they ultimately want to live their lives,
realize the degree of agency that they have individually and
collectively to apply pressure to these companies and hold them
accountable when they do not like what they see, and
ultimately shepherd in a completely different vision for AI development
(15:08):
where people can truly benefit from this technology.
Speaker 2 (15:12):
Karen, it's been wonderful to talk to you. I thank
you so much for your time, really appreciate it.
Speaker 1 (15:18):
For more from the Sunday session with Francesca Rudkin, listen
live to News Talks it B from nine am Sunday,
or follow the podcast on iHeartRadio
You're listening to the Sunday Session podcast with Francesca Rudkin
from News TALKSB So.
Speaker 2 (00:13):
Just what does the future of AI have in store
for us all? None of us really know, do we?
Seven years ago, when investigative journalist Karen Howe started looking
into Sam Oltman's Open AI, she thought they were one
of the good guys. Extensive research and unparalleled access to
those closest to the AI race have left Karen with
a much different view of tech companies and those leading
(00:33):
the charge who work in this space has made her
one of the most foremost tech journals covering AI. She's
been listed in Time Magazine's one hundred Most Influential People,
and Karen is heading to New Zealand for the Auckland
Writers Festival with her book Empire of AI and Karen
Howe's with me now.
Speaker 3 (00:49):
Good morning, Hi Francesca, thank you so much for having me.
Speaker 2 (00:53):
Firstly, and I'm sure my audience knows this, but can
we just define the term AI and AGI?
Speaker 3 (01:00):
Yes? So, I always like to say that the word
AI is like the word transportation. It refers to such
a wide collection of technologies that can be as different
as different from a bicycle to a rocket, and really
these technologies are just loosely tied together based on the
fact that they are each trying to mimic a certain
(01:22):
aspect of human capability. So there are systems that mimic
our language abilities, there are systems that mimic our listening abilities,
are visual recognition and it Ultimately each of these these
different types of AI technologies can actually be vastly different
(01:44):
and have very different cost benefit trade offs. AGI is
a term that refers to artificial general intelligence and is
meant to evoke this idea that somehow the ultimate quest
for building AI technologies is to replicate all human capabilities
(02:05):
in a single system. But one of the problems with
AGI is that it's very poorly defined, in that we
don't really have good ways of defining what human intelligence
is and whether or not we've actually reached the goalposts
of AGI. And so my criticism of companies that use
this kind of rhetoric is that they are essentially just
(02:27):
leveraging the lack of a shared definition as a way
to just hype up their technologies.
Speaker 2 (02:33):
Bringant, thank you for starting us off with it. That's excellent.
Is AGI about science? The betament of humankind, power or
money or all of the above.
Speaker 3 (02:44):
Primarily it's about power and money. AGI at this point
is you could say, almost a quasi religious quest, and
just like other religions, there is this idea that it
is ultimately meant to bring people who believe in to heaven,
(03:07):
and it could also be a tool that damns people
to hell. But the reason why this is such a
potent narrative and is used so often is because companies
that are saying that they're building AGI are ultimately able
to use this to justify the accruval of an enormous
amount of money and power. So all of the tech
(03:29):
companies that are developing AI and using the AGI narrative,
like open Ai, Anthropic, Xai, Google, they now are the
most valuable companies in the world, and they still want
to continue accruing yet more capital to build their technologies.
(03:51):
They are deploying more capital than ever any other industry
ever before to build massive infrastructure projects all around the world. So, yeah,
at the heart of this conversation is indeed money and power,
and we'll.
Speaker 2 (04:08):
Dig into that shortly. But AI open AI is sort
of the center of the book that you've written, and
open AI began as an ultruistic organization. It was a
nonprofit open organization, is implied by the title, It had
this idealistic governance structure. What was its original objective?
Speaker 3 (04:28):
So I have a different answer for this now versus
when I first started covering Open AI. Back then, I
thought they had this kind of pure hearted, mission driven
objective of trying to ensure that AI advancement ultimately goes
well and benefits all of humanity, and that then there
was some kind of profit motive that slowly corrupted the
(04:52):
organization over time. But in hindsight, I think they're real objective.
Their objective that wasn't actually explicitly stated when public messaging
was a desire to become the dominant AI lab in
the world or the dominant force shaping AI technology development
(05:14):
in the world, and you could say that that has
never changed, and what we're seeing now is just a
continued manifestation of this desire for dominance.
Speaker 2 (05:27):
Because the hugely competitive these II companies. This is what
you learn through the book. There is this drive to
be first, these ideological clashes, which makes each player think
that they need to be first to deliver AI for
their purpose because it's the right purpose, and yet that
ambition doesn't quite sit with the ethically minded AI that
they claim to sort of start out with, does it.
Speaker 3 (05:51):
That's exactly right. I mean, the thing that's so interesting
is I often use this analogy that the AI world
is a little bit like Doune The Science Fiction Network
by Frank Herbert. And the reason is because in doing
the main character poly treatise, he knows that there are
(06:11):
these myths that are seated on different planets, including his
own oracus, and that that myth help portrays himself as
the coming of the Messiah, and that if he steps
into the myth, he can leverage it to control the people.
And so he steps into it, but he genuinely believes
that this is to ultimately achieve a good purpose, like
(06:34):
he is doing it in a morally righteous way. And
then he also begins to lose himself in the myth
because every day he's living and embodying this myth, and
the myth and reality start to blur for him, and
he starts to wonder whether or not actually, in fact,
he is the Messiah and the myth is real. And
this is just such an applicable metaphor for how the
(06:58):
AI world works because there's all this myth making about
AGI that many people know is myth making. They are
leveraging it as this marketing rhetoric to accurrue more money
and power to the company. But they also exist in
this space where myth and reality are blurred. They breathe
(07:18):
this myth day in and day out, and they begin
to wonder whether or not it's true, And at the
end of the day they're like, irrespective of whether or
not it's true, both myth making and this quest is
ultimately for good. We are in the moral we are
we are doing something morally righteous. And so there is
(07:39):
this degree of like to say it in a in
like a not too flattering way, self delusion that happens
within the AI world, where people are the revealed pursuit
is of money and power, but their stated pursuit continues
to be this really these like high minded ideals where
(08:02):
if you talk with most people in the AIR world,
they genuinely believe that they are on the quest to
bringing benefit to all of humanity.
Speaker 2 (08:10):
And Karen, who is shaping AI is important, isn't it?
Is it being shaped by too small a pull of people?
Speaker 3 (08:18):
Absolutely? I mean, I think that is the central problem
of the A industry today. And the reason why I
call these companies empires of AI is because they essentially
are these entities that have accrued an enormous amount of power,
and ultimately they are these small number of people are
(08:42):
dictating to billions of people around the world how this
technology that is fundamentally consequential to the future is going
to go, and therefore how everyone in the future is
meant to live and relate to each other and to
work without any input from those billions of people. And
(09:03):
so it doesn't really matter like who is the top
in this kind of scenario, as long as there's this
power structure that enables a small group of people to
dictate these things to billions of people without any accountability,
it really doesn't like just switching out the cast of
characters at the top doesn't really solve the problem.
Speaker 2 (09:26):
It's a very broad question, and I know that it
can't be a quick answer, but you know, maybe you
could just touch on the social and environmental costs of AGI.
Speaker 3 (09:36):
So I don't think people fully understand that AI as
a digital technology has actually a very physical and involved
supply chain that also includes an enormous amount of human labor.
So there are two aspects of AI development that are
(09:58):
often overlooked because of this lack of awareness about that
supply chain. One is that there's an enormous amount of
labor exploitation that happens as part of the supply chain,
and the other is that there's this massive physical manifestation
of AI development, which comes in the form of data
centers and supercomputers that are popping up all around the world.
(10:21):
When it comes to the labor exploitation, I talk in
my book about these content moderators that Opening Eye contracted
in Kenya to build a content moderation filter that they
ultimately put on chat GBT to ensure that users would
never be exposed to the horrific, toxic content that they
(10:44):
were scraping from Reddit and other social media forums to
train their AI models. And in the same way as
content moderators in social media are these content moderators, their
psychological state of mind just completely devolved. And not only
did they break down as individuals, their families broke down,
(11:04):
their communities broke down because they are part of a
fabric in society and when they can't support themselves, there's
suddenly a hole in that fabric, and that fabric begins
to unwind the other aspect, the environmental aspect. People. You know,
people are familiar with data centers. Generally speaking, people understand
(11:26):
data centers to be essential to our modern digitally mediated lives.
But what is not well understood is the degree to
which AI data centers are of a fundamentally different breed
of data center. So Meta's first data center in twenty eleven,
compared to their current AI supercomputer that they are building
(11:46):
in Louisiana, the supercomputer is four hundred times the size
of that first data center, and it is on track
to be one fifth the size of Manhattan and to
eventually use the same amount of power as the average
power demand in all of New York City. So these
(12:07):
facilities are the size of cities and use the power
demand of cities, and there are like they're popping up
all around the world, Like this is just one facility
that I'm talking about, And so the data center expansion
is single handedly beginning to reverse the climate gains that
(12:32):
we made in the last decade, because what are these
facilities actually being powered by. They're being powered by fossil fuels.
The technistry loves to talk this big game about how
they're going to use nuclear to power these facilities with
more clean with clean energy, But the nuclear build out
(12:55):
would not be able to happen nearly fast enough for
supporting this data center build out. And so what we're
actually seeing is coal plants that were meant to be
retired that are instead having their lives extended, natural gas plants,
new natural gas plants that are being built, and there
(13:17):
have been many many cases in which these plants or
these plants end up. You know, they're not just spewing carbon,
they're also spewing air pollutants into communities. And there's an
infamous case of Elon Musk building a giant supercomputer called
(13:37):
Colossus in Memphis, Tennessee, and using thirty five methane gas
turbines to power this facility, and the community that exists
near this facility found out that these methane gas turbines
had popped up overnight when they started smelling what smelled
like a gas leak in their homes, and they suddenly
discovered that they know they were having their right to
(14:02):
clean air taken away from them. So there's just all
of these inner law sucking harms that are very hidden
to the average consumer of AI technologies. When it comes
to this technology's development.
Speaker 2 (14:18):
Karen, AI is moving, it's you predict how it's going
to continue to develop.
Speaker 3 (14:22):
I don't engage in predictions because predictions assume that the
future's already written, and all of my work is ultimately
about how we today, based on our actions, create the
future together. And so what I hope will happen with
the AI industry and with AI development is that everyone
(14:44):
listening to this program, everyone that's thinking about AI and
thinking about how they ultimately want to live their lives,
realize the degree of agency that they have individually and
collectively to apply pressure to these companies and hold them
accountable when they do not like what they see, and
ultimately shepherd in a completely different vision for AI development
(15:08):
where people can truly benefit from this technology.
Speaker 2 (15:12):
Karen, it's been wonderful to talk to you. I thank
you so much for your time, really appreciate it.
Speaker 1 (15:18):
For more from the Sunday session with Francesca Rudkin, listen
live to News Talks it B from nine am Sunday,
or follow the podcast on iHeartRadio
Popular Podcasts
Stuff You Should Know
If you've ever wanted to know about champagne, satanism, the Stonewall Uprising, chaos theory, LSD, El Nino, true crime and Rosa Parks, then look no further. Josh and Chuck have you covered.
The MeatEater Podcast
Building on the belief that a deeper understanding of the natural world enriches all of our lives, host Steven Rinella brings an in-depth and relevant look at all outdoor topics including hunting, fishing, nature, conservation, and wild foods. Filled with humor, irreverence, and things that will surprise the hell out of you, each episode welcomes a diverse group of guests who add their own expertise to the vast world of the outdoors. Part of The MeatEater Podcast Network.
Crime Junkie
Does hearing about a true crime case always leave you scouring the internet for the truth behind the story? Dive into your next mystery with Crime Junkie. Every Monday, join your host Ashley Flowers as she unravels all the details of infamous and underreported true crime cases with her best friend Brit Prawat. From cold cases to missing persons and heroes in our community who seek justice, Crime Junkie is your destination for theories and stories you won’t hear anywhere else. Whether you're a seasoned true crime enthusiast or new to the genre, you'll find yourself on the edge of your seat awaiting a new episode every Monday. If you can never get enough true crime... Congratulations, you’ve found your people. Follow to join a community of Crime Junkies! Crime Junkie is presented by Audiochuck Media Company.