February 6, 2025 • 8 mins
Can AI revolutionize the way we think about energy consumption and power grid demands? Venture into the world of cutting-edge AI technology with us as we examine the projected 125,000 megawatt surge in data center electricity demand, driven not just by traditional needs but also by the expanding crypto universe. We uncover the breakthrough of DeepSeek, China's open-source AI model that claims to match the performance of leading proprietary models at a fraction of the energy and cost. This development shook Wall Street, as tech giants like NVIDIA faced significant market value losses, sparking intense scrutiny over the legitimacy of DeepSeek's extraordinary claims.
Join us for a thorough exploration of DeepSeek's potential to change the AI landscape and its implications for future energy consumption. We explore whether this could lead to Jeevan's paradox, where cheaper computational capabilities result in increased usage. Despite the mixed reviews and questions about its creative task limitations, DeepSeek's impact is undeniable, particularly in inference and real-time decision-making. We unpack these concepts, sharing insights into how AI's ability to recognize patterns is reshaping real-world applications. This episode offers a captivating look at the intersection of technology and energy, a must-listen for anyone curious about the future of AI.
Mark as Played
Transcript
Episode Transcript
Available transcripts are automatically generated. Complete accuracy is not guaranteed.
Speaker 1 (00:00):
Just when you get comfortable thinking you know
something, you find out thatmaybe you don't.
In a series of videos late lastyear, I addressed the issue of
exploding data centerelectricity demand and the
enormous number of applicationsutilities had received.
In recent months I've beentracking these in a spreadsheet
based on various press articlesand releases in trade press and
thus far I've got over 125,000megawatts of new projected data
(00:24):
center demand.
Not all of this demand isAI-related.
Some new load will serve yourtypical data center applications
, while some may even be servingcrypto loads, now that crypto's
in fashion in Washington.
But there have been some subtlesigns that perhaps this new
load might not be as big asheadlines suggest.
Skepticism was already the wordof the day before news came out
(00:45):
of China last week that anopen-source AI large language
model, an LLM there calledDeepSeek, was nearly as good as
some of the proprietary modelsbeing built here in the US by
some of the biggest players inthe space.
The news that mattered most tomarkets was that it was not only
competitive but much cheaper,using fewer chips and far less
(01:05):
power.
Deepseek reported that itsmodel took only two months and
less than six million dollars tobuild using a less advanced and
less costly H800 NVIDIA chip.
The one-day carnage on WallStreet was amazing to behold.
Leading chip maker NVIDIA'sshare price fell off a cliff,
losing 17% and $600 billion witha B of market value.
(01:27):
Modular nuclear and fuel cellstocks got savage as well,
shedding up to 25% of theirstock prices.
They quickly rebounded, though,as over the ensuing week,
additional news filtered outthat perhaps those numbers
weren't quite so reliable,coupled with accusations that
there'd been some so-calleddistilling, ie transferring
(01:50):
knowledge from open AI toDeepSeq or at least some reverse
engineering from other AImodels.
So it wasn't like DeepSeq wasbuilt from scratch.
Now come three questions relatedto the grid and future power
consumption.
First, how much of DeepSeq'sclaims will eventually prove to
be true, both in terms of thetime and resources to build the
ELLM and the implications interms of power and what the
other large language modelsessentially need for chips and
(02:14):
power, as they've been bruteforcing their way through their
trainings?
Second, is the model reallythat good?
If one can really build AIcapabilities more cheaply, does
that in fact lead to Jeevan'sparadox, ie, the less expensive
the computational capacity is,the more of it we'll use?
As far as the first claim, thatremains to be verified.
(02:34):
However, if it's remotely true,it could dramatically change
how much that currentenergy-intensive brute force
huge chip approach is applied toLLM model development in the
future.
That would bring down energyconsumption figures way down,
though nobody knows by quite howmuch.
This is still all too new.
The second claim also may notstand up to further scrutiny.
(02:55):
As noted, some anecdotalevidence I've seen and others
have seen, suggests thatDeepSeek is not really that good
at answering some simplequestions, and OpenAI has made
some claims that need to beverified.
But what is true?
The model is pretty good.
A New York Times tech reporterthat spent half of the Monday a
week ago playing with the techcame away impressed, noting that
it compared well with OpenAI'sChatGPT and Anthropix CLOD.
(03:19):
It solved some complex math,physics and reasoning problems
at twice the speed of chat, andits responses to computer
programming questions were quoteas in-depth and speedy as its
competitors.
It wasn't quite so good atcomposing poetry, planning
vacations or coming up withrecipes, but so what If it's
almost as good at a fraction ofthe price?
(03:39):
Well, so there, it looks likethere's a there there.
The next question then comesdown to use, or so-called
inference DeepSeek is free andit was the most frequently
loaded app two weeks ago.
As defined by Perplexity, aiquote inference involves using
the patterns and relationshipslearned during training to solve
(04:01):
real-world tasks withoutfurther learning.
For instance, a self-drivingcar recognizing a stop sign on
an unfamiliar road is an exampleof inference.
Unquote provision of thatresponse to my query that I did
with perplexity was also anexample of inference.
See what I did there.
Inference can also help withreal-time decision making and
involves a number of steps First, data preparation.
(04:23):
Second, model loading.
Third, processing andprediction.
Fourth, output generation togive you the information or the
results you seek.
Inference is very energyintensive, so if we use less
energy on LLMs but they getcheaper and more ubiquitous,
what does that mean for energyconsumption?
In the arena of inference,we're so early into the
(04:44):
adaptation and adoption of thesetools that nobody knows.
But as far as the electricityrequired, we could be in the
midst of a typical Gardner hypecycle, such as the one we
experienced in the early 90sdot-com frenzy when petcom's
sock puppet was going todominate the dog food industry.
Admittedly, 25% of DominionEnergy's demand in Virginia is
(05:05):
already dedicated to servingdata centers, and AI will
clearly have many uses, some ofwhich we can only imagine today.
So we'll certainly see moreenergy use, but the LLMs may run
into various limits withdeclining economies of scale
that would eventually reduceexpected demand.
There'll also be substantialgains in processing and cooling
efficiencies that drive energyrequirements down, and we will
(05:27):
probably see those results inthe years to come.
Right now, we're still in thevery early days of throwing
money, a first version of chipsand data, at the opportunity,
but checkbooks and coffers arenot unlimited and a focus on
efficiency will inevitablyfollow it always does.
There will also be companiesthat don't survive the race.
That will probably be dominatedby only a few deep pocket
(05:48):
participants, although scrappy,low-budget startups like
DeepSeek suggest that perhaps anoligarchy isn't inevitable.
But if this does go the sameway the search engine race did,
we'll be left with only a smallnumber of well-resourced players
, and this LLM quest may yieldsimilar results, with most
companies failing or beingconsolidated, and if you don't
(06:08):
believe me, you can go askJeeves.
There's also a big issue relatedto these headline demand
numbers.
The data companies may befiling many more applications
with utility for supply thanthey intend to actually develop
because of the way the processfor interconnection with
utilities actually works.
Only a small number ofutilities actually have rigorous
(06:29):
procedures for evaluating theapplications to ensure they're
likely to get to physicalservice.
The best ones, like seasonedveteran Dominion Energy, require
proof of control of land, afinancial commitment from the
data company to support requiredengineering studies and
signature of a constructionletter of authorization
obligating the obligant to payfor all project-related
(06:50):
expenditures, regardless ofwhether the project breaks
ground.
Only then does an electricservice agreement, an ESA, get
signed that makes its way intothe forecast.
A review of various forecastsin other parts of the country
demonstrate that this same levelof rigor is not routinely
applied.
Thus it's quite likely thatdata companies are submitting
multiple interconnectionrequests and utilities are
(07:11):
over-reporting the capacitynumbers.
Many data companies are likelydoing what you and I would do.
If we needed lots of views asfast as possible, we'd logically
submit multiple applications tonumerous utilities with the
hopes that at least some ofthose would get to.
Yes, it's not possible to gaininsight into exactly what's
happening at any point in time,since the industry's competitive
(07:32):
remains a high degree ofconfidentiality, but it's very
likely there are numerousplaceholder phantom requests.
The analog on the bulk powersupply side of the industry may
be instructive where over 10,000generation projects wait in
transmission interconnectionqueues and, if recent history is
a guide, fewer than 20% ofthose endeavors will actually
(07:54):
get built.
If utilities further tighten uptheir load interconnection
requirements and implement morerigorous procedures that require
higher upfront financialcommitments, we may get a better
sense as to how many realapplications are out there.
It's clear that AI has realvalue to society and we are now
beginning to see some use casesemerge.
It's also clear we're in thevery early days, with rapidly
(08:16):
evolving technologies andbusiness models and many
unanswered questions.
However, getting past thecurrent hype cycle will take
some time.
We won't know the fullimplications until we start to
see some projects proceed whileothers are canceled.
And if you don't believe me,ask Perplexity AI, it tells me.
Quote several factors suggestthat only a fraction of the
(08:38):
proposed projects will likely becompleted.
Unquote.
Amen to that.
Thanks for watching and we'llsee you again, hopefully next
week.
Just when you get comfortable thinking you know
something, you find out thatmaybe you don't.
In a series of videos late lastyear, I addressed the issue of
exploding data centerelectricity demand and the
enormous number of applicationsutilities had received.
In recent months I've beentracking these in a spreadsheet
based on various press articlesand releases in trade press and
thus far I've got over 125,000megawatts of new projected data
(00:24):
center demand.
Not all of this demand isAI-related.
Some new load will serve yourtypical data center applications
, while some may even be servingcrypto loads, now that crypto's
in fashion in Washington.
But there have been some subtlesigns that perhaps this new
load might not be as big asheadlines suggest.
Skepticism was already the wordof the day before news came out
(00:45):
of China last week that anopen-source AI large language
model, an LLM there calledDeepSeek, was nearly as good as
some of the proprietary modelsbeing built here in the US by
some of the biggest players inthe space.
The news that mattered most tomarkets was that it was not only
competitive but much cheaper,using fewer chips and far less
(01:05):
power.
Deepseek reported that itsmodel took only two months and
less than six million dollars tobuild using a less advanced and
less costly H800 NVIDIA chip.
The one-day carnage on WallStreet was amazing to behold.
Leading chip maker NVIDIA'sshare price fell off a cliff,
losing 17% and $600 billion witha B of market value.
(01:27):
Modular nuclear and fuel cellstocks got savage as well,
shedding up to 25% of theirstock prices.
They quickly rebounded, though,as over the ensuing week,
additional news filtered outthat perhaps those numbers
weren't quite so reliable,coupled with accusations that
there'd been some so-calleddistilling, ie transferring
(01:50):
knowledge from open AI toDeepSeq or at least some reverse
engineering from other AImodels.
So it wasn't like DeepSeq wasbuilt from scratch.
Now come three questions relatedto the grid and future power
consumption.
First, how much of DeepSeq'sclaims will eventually prove to
be true, both in terms of thetime and resources to build the
ELLM and the implications interms of power and what the
other large language modelsessentially need for chips and
(02:14):
power, as they've been bruteforcing their way through their
trainings?
Second, is the model reallythat good?
If one can really build AIcapabilities more cheaply, does
that in fact lead to Jeevan'sparadox, ie, the less expensive
the computational capacity is,the more of it we'll use?
As far as the first claim, thatremains to be verified.
(02:34):
However, if it's remotely true,it could dramatically change
how much that currentenergy-intensive brute force
huge chip approach is applied toLLM model development in the
future.
That would bring down energyconsumption figures way down,
though nobody knows by quite howmuch.
This is still all too new.
The second claim also may notstand up to further scrutiny.
(02:55):
As noted, some anecdotalevidence I've seen and others
have seen, suggests thatDeepSeek is not really that good
at answering some simplequestions, and OpenAI has made
some claims that need to beverified.
But what is true?
The model is pretty good.
A New York Times tech reporterthat spent half of the Monday a
week ago playing with the techcame away impressed, noting that
it compared well with OpenAI'sChatGPT and Anthropix CLOD.
(03:19):
It solved some complex math,physics and reasoning problems
at twice the speed of chat, andits responses to computer
programming questions were quoteas in-depth and speedy as its
competitors.
It wasn't quite so good atcomposing poetry, planning
vacations or coming up withrecipes, but so what If it's
almost as good at a fraction ofthe price?
(03:39):
Well, so there, it looks likethere's a there there.
The next question then comesdown to use, or so-called
inference DeepSeek is free andit was the most frequently
loaded app two weeks ago.
As defined by Perplexity, aiquote inference involves using
the patterns and relationshipslearned during training to solve
(04:01):
real-world tasks withoutfurther learning.
For instance, a self-drivingcar recognizing a stop sign on
an unfamiliar road is an exampleof inference.
Unquote provision of thatresponse to my query that I did
with perplexity was also anexample of inference.
See what I did there.
Inference can also help withreal-time decision making and
involves a number of steps First, data preparation.
(04:23):
Second, model loading.
Third, processing andprediction.
Fourth, output generation togive you the information or the
results you seek.
Inference is very energyintensive, so if we use less
energy on LLMs but they getcheaper and more ubiquitous,
what does that mean for energyconsumption?
In the arena of inference,we're so early into the
(04:44):
adaptation and adoption of thesetools that nobody knows.
But as far as the electricityrequired, we could be in the
midst of a typical Gardner hypecycle, such as the one we
experienced in the early 90sdot-com frenzy when petcom's
sock puppet was going todominate the dog food industry.
Admittedly, 25% of DominionEnergy's demand in Virginia is
(05:05):
already dedicated to servingdata centers, and AI will
clearly have many uses, some ofwhich we can only imagine today.
So we'll certainly see moreenergy use, but the LLMs may run
into various limits withdeclining economies of scale
that would eventually reduceexpected demand.
There'll also be substantialgains in processing and cooling
efficiencies that drive energyrequirements down, and we will
(05:27):
probably see those results inthe years to come.
Right now, we're still in thevery early days of throwing
money, a first version of chipsand data, at the opportunity,
but checkbooks and coffers arenot unlimited and a focus on
efficiency will inevitablyfollow it always does.
There will also be companiesthat don't survive the race.
That will probably be dominatedby only a few deep pocket
(05:48):
participants, although scrappy,low-budget startups like
DeepSeek suggest that perhaps anoligarchy isn't inevitable.
But if this does go the sameway the search engine race did,
we'll be left with only a smallnumber of well-resourced players
, and this LLM quest may yieldsimilar results, with most
companies failing or beingconsolidated, and if you don't
(06:08):
believe me, you can go askJeeves.
There's also a big issue relatedto these headline demand
numbers.
The data companies may befiling many more applications
with utility for supply thanthey intend to actually develop
because of the way the processfor interconnection with
utilities actually works.
Only a small number ofutilities actually have rigorous
(06:29):
procedures for evaluating theapplications to ensure they're
likely to get to physicalservice.
The best ones, like seasonedveteran Dominion Energy, require
proof of control of land, afinancial commitment from the
data company to support requiredengineering studies and
signature of a constructionletter of authorization
obligating the obligant to payfor all project-related
(06:50):
expenditures, regardless ofwhether the project breaks
ground.
Only then does an electricservice agreement, an ESA, get
signed that makes its way intothe forecast.
A review of various forecastsin other parts of the country
demonstrate that this same levelof rigor is not routinely
applied.
Thus it's quite likely thatdata companies are submitting
multiple interconnectionrequests and utilities are
(07:11):
over-reporting the capacitynumbers.
Many data companies are likelydoing what you and I would do.
If we needed lots of views asfast as possible, we'd logically
submit multiple applications tonumerous utilities with the
hopes that at least some ofthose would get to.
Yes, it's not possible to gaininsight into exactly what's
happening at any point in time,since the industry's competitive
(07:32):
remains a high degree ofconfidentiality, but it's very
likely there are numerousplaceholder phantom requests.
The analog on the bulk powersupply side of the industry may
be instructive where over 10,000generation projects wait in
transmission interconnectionqueues and, if recent history is
a guide, fewer than 20% ofthose endeavors will actually
(07:54):
get built.
If utilities further tighten uptheir load interconnection
requirements and implement morerigorous procedures that require
higher upfront financialcommitments, we may get a better
sense as to how many realapplications are out there.
It's clear that AI has realvalue to society and we are now
beginning to see some use casesemerge.
It's also clear we're in thevery early days, with rapidly
(08:16):
evolving technologies andbusiness models and many
unanswered questions.
However, getting past thecurrent hype cycle will take
some time.
We won't know the fullimplications until we start to
see some projects proceed whileothers are canceled.
And if you don't believe me,ask Perplexity AI, it tells me.
Quote several factors suggestthat only a fraction of the
(08:38):
proposed projects will likely becompleted.
Unquote.
Amen to that.
Thanks for watching and we'llsee you again, hopefully next
week.
Popular Podcasts
Dateline NBC
Current and classic episodes, featuring compelling true-crime mysteries, powerful documentaries and in-depth investigations. Follow now to get the latest episodes of Dateline NBC completely free, or subscribe to Dateline Premium for ad-free listening and exclusive bonus content: DatelinePremium.com
24/7 News: The Latest
The latest news in 4 minutes updated every hour, every day.
Therapy Gecko
An unlicensed lizard psychologist travels the universe talking to strangers about absolutely nothing. TO CALL THE GECKO: follow me on https://www.twitch.tv/lyleforever to get a notification for when I am taking calls. I am usually live Mondays, Wednesdays, and Fridays but lately a lot of other times too. I am a gecko.