May 7, 2025 • 46 mins
Where should your AI workloads run? It's one of the most overlooked questions in AI strategy. From surprising constraints around power, cooling and floor space, to the growing demand for GPU-as-a-Service models, this episode delivers a field-level view of the challenges enterprises face when moving from AI proof of concept to production. You’ll hear why infrastructure readiness assessments are essential, how AI workloads differ from traditional IT, and what to consider before buying that next GPU cluster.
Support for this episode provided by: Equinix
Learn more about this weeks guests:
Chris Campbell is Sr. Director of AI Solutions at WWT, overseeing AIaaS/GPUaaS, Facilities & Infrastructure strategy and delivery. He's held leadership roles in executive engagement, engineering, and architecture at WWT, and led practices at Forsythe, Red Hat, BEA Systems, and AT&T. Chris holds a BA from Columbia University and an MBA from the University of Maryland, where he was a Dingman Entrepreneur Scholar.
Don Molaro is a tech professional focused on making data centers carbon neutral. Based in Cupertino, he holds a master's in Computer Science and 33 U.S. patents. He specializes in systems programming, high-performance storage, and software-defined systems, with experience across industries from finance to HPC. Previously, he held senior roles at Citibank, DataDirect Networks, and Hitachi Data Systems.
Chris and Don's top pick: What is GPU-as-a-Service (GPUaaS) or GPU Cloud?
The AI Proving Ground Podcast leverages the deep AI technical and business expertise from within World Wide Technology's one-of-a-kind AI Proving Ground, which provides unrivaled access to the world's leading AI technologies. This unique lab environment accelerates your ability to learn about, test, train and implement AI solutions.
Learn more about WWT's AI Proving Ground.
The AI Proving Ground is a composable lab environment that features the latest high-performance infrastructure and reference architectures from the world's leading AI companies, such as NVIDIA, Cisco, Dell, F5, AMD, Intel and others.
Developed within our Advanced Technology Center (ATC), this one-of-a-kind lab environment empowers IT teams to evaluate and test AI infrastructure, software and solutions for efficacy, scalability and flexibility — all under one roof. The AI Proving Ground provides visibility into data flows across the entire development pipeline, enabling more informed decision-making while safeguarding production environments.
Mark as Played
Transcript
Episode Transcript
Available transcripts are automatically generated. Complete accuracy is not guaranteed.
Speaker 1 (00:00):
AI's insatiable hunger for power is reshaping
the global energy map, and fast.
Just last month, theInternational Energy Agency
warned that data centerelectricity use will double by
the year 2030, with AIresponsible for most of that
surge.
Reuters put an even sharperpoint on it by the end of the
decade, one in eight USelectrons could be flowing into
(00:22):
server halls instead of homes orfactories.
One in eight US electrons couldbe flowing into server halls
instead of homes or factories.
Now, that's the landscape.
But our two guests today, ChrisCampbell, a senior director of
AI solutions, and Don Malero, achief technical advisor, both of
which are colleagues of minehere at WWT, are here to talk
about the reality that today'sAI infrastructure decisions
aren't just technical, they'reoperational and they're
(00:44):
financial out their core.
Now you might be asking why doesall of this matter?
Well, the competitive edge in2025 isn't just having AI, it's
running it where it performsbest, at cost and under a risk
profile you're comfortable with.
Get that calculus wrong and thesame model that promised to cut
fraud or discover a new drugnow becomes a drag.
(01:04):
So stick around.
In a few minutes, don willshare a quietly powerful rule of
thumb he uses before anydeployment for whether a
workload belongs on-prem, incolo or in the cloud.
It's simple, it's actionableand it might save you more money
than the GPU's cost in thefirst place.
This is the AI Proving Groundpodcast from Worldwide
Technology.
Let's get to it, chris.
(01:35):
I get the sense from speakingto either clients or hearing
from experts that we have hereat Worldwide Technology, that
consideration of AI workloadsand how they're deployed can
sometimes maybe even often be anafterthought.
Is that an accurate assessment,and what are you seeing from
that perspective?
Speaker 2 (01:54):
Yeah, there's kind of a few things we're seeing is
there was a lot of folks thatwere doing what I'll call stroke
of the pen type purchases ofequipment, and so they would buy
this equipment without reallythinking about how they were
going to support it from aninfrastructure standpoint, power
standpoint, data centerstandpoint.
So now they're looking at itafter the fact and saying, well,
(02:15):
where should this stuffactually go?
And in some cases they may notbe able to support it from their
data center.
We've seen that in quite a fewdifferent people and really from
our standpoint we say, well,let's go ahead and try to plan
ahead.
So we're really encouraging ourclients now to think ahead.
We do a facilities assessmentand this will land in one of a
few places.
(02:36):
But with that planning, I thinkthat'll help them to really be
able to deploy this in ameaningful way without any
slowdown or anything that'sgoing to happen with them to be
able to delay these projects.
Speaker 1 (02:49):
Yeah, don you know, for those that maybe did treat
it a little bit as anafterthought or thinking that it
would just kind of take care ofitself, what are the stumbling
blocks for them if they don'ttake this into consideration
from the get-go?
Speaker 3 (03:04):
Well, I mean in the facilities business, there's the
iron triangle, which is atriangle between power, space
and cooling.
The first stumbling block thatwe see organizations run into is
power.
They realize that, you know,with an IT rack that, on the
(03:29):
high end, is drawing in the 17to 20 kilowatt range, and
they're talking about deployingan AI rack that starts at 50
kilowatts and can conceivably goup to well over, well over 100,
into like the 130, 150 range.
Right now, power becomes thefirst thing that they have to
(03:54):
deal with Very quickly, and it'sa different kind of power.
We're going from 208 volt powerin a traditional data center to
most of the ai systems arerequiring 415 volt power.
So there's not only is it a lotmore power, it's a different
kind of power.
(04:14):
Um, the second piece to thatthat comes along is cooling.
Obviously, uh, the second lawof thermodynamics you know it
still applies.
Uh, the power you put into therack has to be extracted somehow
.
So cooling becomes the nextissue that organizations and
(04:36):
customers start to deal with.
There's lots of different waysto do cooling, and we can talk
about that at some length, butanother piece of it that comes
along very quickly on top ofthat is the space itself.
Traditional raised floor datacenter raised floor environments
simply can't support the weightof modern AI systems about
(05:08):
things that are well over 3,000pounds going into a data center
that might be built for 1,200pound racks.
So we're dealing with things atthe maximum.
So we're dealing withchallenges on power, space and
cooling.
The final challenge that getsput onto this is the cabling and
the amount of interconnect thatthese systems require, and
(05:29):
we've got lots of issues aroundstructured cabling that goes
into these systems.
That is orders of magnitudelarger than you would get in an
IT workload.
So it's power space, coolingcabling, and that's just to get
(05:51):
to day one operation.
So there's lots of challengesall over the map for this.
Speaker 1 (05:57):
Yeah, I like that iron triangle.
Did you come up with that onyour own?
Is that an overpriced area?
Speaker 3 (06:03):
That's a traditional thing in the facilities business
that certainly my colleaguesBruce and Mike have been putting
up in front of customers for along time.
I'm relatively new to thehardcore facilities end of the
business.
I come at it from a compute andan IT operational point of view
(06:28):
and it's been a real educationworking with the facilities
infrastructure team and thedepth of knowledge that they
have.
But yeah, I can't take creditfor that.
I'm going to give full creditto Bruce Gray and Mike Parham on
that, for teaching me aboutfacilities, Sure, sure.
Speaker 1 (06:52):
Well, chris, maybe take a step back for us here and
articulate, just for those thatmay not be familiar, what
exactly AI workload means forour enterprise customers.
What exactly AI workload meansfor our enterprise customers?
Is it different fromtraditional IT workloads that
we've been talking about overthe past, however many decades?
Speaker 2 (07:14):
And if it is different, why is it different?
Yeah, you know this is not yourfather's AI workload or
infrastructure workload thatthey have.
It requires a fully different,as Don alluded to different
setup in the data center,different amounts of power, but
the workload itself really isyou kind of consider what folks
are trying to accomplish.
So they start with a use case,they have some data that they
(07:37):
use to point to, and then theyuse a training model within AI
to be able to go and train it,and that training model requires
a lot of compute power.
That's why NVIDIA is so frontand center in this is that
they've got these chips andallows them to really process
this in a really fast way.
That's why, when you use aGPT-4 or some other kind of
(07:59):
chatbot, it reacts so quicklyand this is why they've got such
a dominant place in theindustry.
But from a workload standpoint,that workload is how are they
processing that informationrapidly to get the answer that
they need to?
And so ultimately, when ittrains, it requires a much
stronger and more powerfulinfrastructure to run on.
(08:20):
But once they can move toinferencing, which means that
they're actually processing on amore long level, and it's
actually what they're doing,from the time they train it
through the rest of its lifespan, is inferencing.
That requires a slightly lesspowerful infrastructure, and you
could do it in a variety ofways, maybe even using different
(08:41):
types of infrastructure for itthat allows it to be able to run
and not require that power.
So it is very different.
This is why we're seeing thisrapid shift in the data center
space to try to find spaces forthese super pods that NVIDIA is
putting out there and otherplaces, because they really
can't fit in any other type oftraditional data center and in
(09:02):
many cases, the legacy datacenters that they have can't
even be retrofitted with therequirements needed to supply it
.
So it's vastly different thanwhat it has been in the past.
Speaker 1 (09:22):
Yeah, and we'll get into talking about where these
workloads can and should run,but Don real quick as well.
Are all AI workloads createdequal?
I'm assuming that there's alevel of nuance to all of this,
and what are some of thoseconsiderations that we should be
thinking about, that ourclients should be thinking about
as they're assessing their AIworkloads and then as we start
(09:44):
to think about where they shouldrun?
Speaker 3 (09:48):
In some way AI workloads are created equal and
in many other ways they're not.
They're all pushing the limit.
They're equal in the sense thatthey are pushing the limits of
what is capable in the industryat the moment, whether that's
(10:11):
leaning on what the computeengine is and quite often NVIDIA
GPUs, the networkinginfrastructure that is going
with that.
There are a lot of very similarconsiderations between
organizations the latency, thelove, you know, the latency, the
bandwidth that happens on thenetwork side and on the storage
side, you know, in making surethat the right data is in the
(10:33):
right place at the right time.
Those are all equally importantconsiderations that
organizations have to look atand that and there are common or
reference solutions to to allof those problems and that's
sort of what we do in the AIPGis is we have those reference
(10:55):
designs, we have them built out,we have good examples of what,
of, of what proper referencearchitecture looks like for
those solutions.
Speaker 1 (11:06):
Yeah, aipg being the AI proving ground that we have
here at WWT to help our clientsgo through their AI journey.
Chris Don mentioned at leastseveral of them computing
storage.
What about data sensitivity,security, latency, things like
that?
What else do we need to beconsidering?
Speaker 2 (11:28):
Yeah, I would say that.
You know, depending upon whatthey're trying to accomplish and
the sensitivity of that data,certainly you have to consider
that and you know that's trulysomething that, as we're looking
and working with some of theearly adopters, they do not want
this data to be any place otherthan an on-prem environment or
a private environment, and so Ithink that as we continue to
(11:48):
progress and more use cases arereleased and they have less
sensitivity about that data,you're going to see other areas,
maybe the cloud or other placesthat'll go, but that data
sensitivity is certainly anissue and then in a location and
I think what we're seeing nowis you could train this in one
area and then ultimately, as yougo out to do the inferencing,
(12:10):
you may want to move it withmore proximity to a cloud source
or other sources so it can runmore effectively and efficiency
and not have that latency.
Speaker 1 (12:20):
And where does considerations of the use case
come in, if at all?
Speaker 2 (12:27):
Well, the use case will depend on how quickly they
want that information in manycases.
So you know what we're seeingin, for example, finance or
we're seeing in health care.
Those are really the earlyadopters that we're seeing.
Life sciences are doing that aswell as some of the retail, and
so that proximity and speed tothat information, depending upon
(12:48):
how quickly they want to get itlet's just say it's fraud
protection that they might beusing within a financial
institution They'll want thatresponse time to be very, very
fast.
And Don's certainly an expertin that area and can jump on and
add to whatever I might bemissing, but I think that's
something they consider as well.
Speaker 1 (13:06):
Yeah, don anything to add there.
Speaker 3 (13:08):
Sure, chris is exactly right.
The underlying infrastructureis important, but the use case
and the business value that goeson top of it is what the
customers are ultimatelyinvesting in.
They're looking at thatbusiness outcome and that's
(13:29):
where all of the other layers ofan AI solution whether that's
learning or whether that'sinferencing or even traditional
AI ML that's what matters to thecustomer.
The infrastructure piece of itis simply a means to an end.
So we have to be we're verycareful about how we build out
(13:53):
those environments whetherthat's a retrofit on premise,
because sometimes it might befor very modest footprints, but
we've seen that some one of thefinancial institutions I'm
working with they're doing.
They're doing theirinferencing-prem because they
can't eke out enough power andcooling in their existing data
(14:14):
center to do it.
For other organizations, they'rebuilding and retrofitting
dedicated environment relativelysmall compared to their overall
footprint, but still dedicatedand reasonably significant
environments in their own datacenters.
Or we're working with customerswho are placing workloads in
(14:38):
co-locators, and some of our keyco-locator partners there are
adapting their environments forvery high scale, very high load
AI type deployments.
And finally, we're looking in,as well as the cloud, the, the
the old cloud service providers,we're looking at new ones that
(15:02):
are specifically dedicated toproviding GPU and high
performance computing in asecure environment for
organizations that have thathave those requirements.
Chris is exactly right.
These workloads are.
They are core to the businessesthat we're dealing with.
(15:24):
They are quite often some ofthe most important workflows and
applications that they'rerunning.
You can, fraud detection is acritical workload at a bank, and
so deploying that and doing itcorrectly and doing it
responsibly is absolutelycritical for these organizations
(15:44):
, and they're having a really,really hard time doing it on
premise with modern computingequipment.
Speaker 1 (15:50):
Really really hard time doing it on premise with
modern computing equipment.
Well, chris, we mentioned thoseoptions there Colo, cloud,
on-prem.
What are you seeing in themarket right now in terms of the
most appropriate way to divvyall that up?
Speaker 2 (16:03):
So the answer is it depends.
I think if you, right now,there's three options that
people are really looking at,and if you look at what some of
the analysts say, and they'retalking about a third of them
being on-prem, a third of thembeing in the cloud and a third
of them being in some kind ofprivate hosted environment like
an AI as a service or a GPU as aservice, which is a leased GPU
(16:28):
environment, and ideally, Ithink that middle area, that
last area I talked about, whichis the GPU as a service, private
hosting is going to be far moreprominent.
The challenges that they faceare if you want to go on-prem,
your data center may not beready.
And so, to Don's point, some ofthese folks might be eking out
inferencing that they have, andthey might be doing that to do
(16:48):
the best they can.
In other cases, they're saying,well, maybe we can try to
retrofit it, but pushing it tothe cloud or pushing it to a
hosting provider like an Equinixor a digital realty, I think,
makes a little bit more sense insome of these cases.
Now, you're not going to go putin the cloud if you've got some
kind of sensitivity to the data.
So that idea that this privatehosting environment is going to
(17:10):
be something that's prominentfor some of these workloads, I
think is very true.
But as these early adopters comeon, some of the easy button is
well, we're going to just go anddo this in the cloud We've
already got an AWS account orthey're going to go ahead and
buy something and put it in aprivate hosted environment
because they're going to make alonger term commitment.
But the GPU as a serviceproviders that I mentioned is
(17:32):
something where, if you want ashort term testing and we're
seeing this with our partnerslike CoreWeave and Scott Data
Center, applied Digital they allare getting a lot of business
right now and people just usingshort term testing so they can
prove out some type of use casethat they're working on or be
able to go get fundings, muchlike they're doing in our AIPG,
(17:52):
but with their own data.
So it's a really interestingtime right now to see this.
But as this business continuesto build over time, I think
you're going to see thesedecisions being made and while
folks will invest in HPA,on-prem, high-performance
architecture on prem, I do thinkthat some of the easy buttons
for them, instead of building adata center, buying $20 million
(18:14):
worth of gear is to figure out aplace to go and host it or to
lease it from someplace.
It's kind of that build versusbuy discussion that people have.
Speaker 1 (18:23):
Yeah, Don.
What do you make of that GPU asa service angle, and doesn't
that also just run the risk ofpotentially running up the cost?
If people are looking to youknow, take advantage of that
scarcity.
Speaker 3 (18:36):
So.
So there's there's as with allthese solutions, there's a I
mean, there's, there's a lot ofnuance in the actual solution.
So, even even when people talkabout GPU as a service, you'll
have environments.
You'll be able to go to a GPUas a service provider, and
(18:59):
there's lots of them that we'repartnering with Scott Data
Center, coreweave, lambda Labs,Applied Digital there are lots
of them in this space.
They all have slightlydifferent product offerings and
dealing with them and gettingyour workload working there is
(19:19):
non-trivial.
For example, they may have adebit.
You may be in a situation whereyou require a cage, dedicated
environment that even the GPU asa service provider doesn't have
access to, because you want torun confidential workloads in
there.
That is some of the offeringsfrom this.
(19:40):
There's much more traditional,even all the way down to
virtualized environments whereyou're multi-tenanting on
hardware with other corporationsor other clients of those
organizations.
So even the offerings thatyou'll see from a GPU as a
(20:00):
service provider, there's a lotof nuance there.
There's also a lot of nuanceabout the level of service
you're going to get from them,which is very different than
what you would get in a cloudservice provider.
Cloud service provider is goingto provide you a complete
end-to-end solution wherethey're going to manage all of
the infrastructure, all of theservices and everything that
goes with that In a GPU.
(20:23):
As a service provider, theymight just be providing you the
bare metal, or they might beproviding you up to the
operating system level, or theymight be providing you up to the
operating system level, or theymight be providing services up
to the Kubernetes and Slurmlevel, or even beyond that.
And this is why I think there'sa lot of value in a customer
coming to WWT, partnering withboth WWT and the GPU as a
(20:48):
service provider to provide anend-to-end solution that
includes access to the hardware,access to a high-performance
computing environment that hasthe right services all the way
up through the application.
And this is where it's a realcase of one plus one equals a
(21:09):
lot more than two for thecustomer.
Because that's where the valueof the partnership is, between
WWT and our GPU as a serviceprovider partners to the
customers.
I think it's not only is itjust a matter of having a place
to run your workloads, it'sactually, you know, we really
(21:33):
believe that you can execute abetter solution for the customer
and those kinds of environments.
Speaker 2 (21:39):
Yeah, brian, I'm going to plus one what Don was
talking about here, but reallytalk about.
You know the investment peoplehave to make.
So if you're asking, if you're aCIO or you're running a center
of excellence at a client andyou're trying to go ask for $5,
$10, $15 million to go to a usecase, they are sometimes wanting
to prove that out before theyget approval from their board on
(22:00):
that cost.
So, using a GPU as a serviceprovider that already has that
equipment in place we see thisagain in our AIPG as well, in
our AI proving ground, wherefolks will come in and just want
to run it on one of the 15reference architectures that we
have.
Well, that's kind of a firststep.
And then, when you really wantto use your own data rather than
some kind of synthetic datajust we use in the AIPG, that's
(22:23):
where they could go into a GPUas a service provider, really
prove this out and determinewhether or not it works, and I
think that's something thatsaves them a lot of time.
It probably saves them somemoney in the initial stages and
so you know, we see a reallyhigh level of use coming for
this GPU as a service as folkscontinue to put their budgets
toward AI projects.
Speaker 3 (22:45):
This episode is supported by Equinix.
Equinix connects businessesglobally through its extensive
network of data centers,enabling seamless digital
interactions.
Expand your reach withEquinix's interconnected
infrastructure.
Speaker 1 (23:06):
What's the market look like?
I heard a story the other daytalking about how there's people
out there looking to build anetwork of folks that just have
excess capacity of their GPUs,thinking like kids in their
parents' basement playing on aGPU card in a video game.
Obviously that's kind of an offangle of that area, but is that
(23:27):
market going to continue toexpand and grow or is it going
to be tight?
Speaker 2 (23:32):
area.
But is that market going tocontinue to expand and grow or
is it going to be tight?
Yeah, the biggest thing thatwe're seeing for GPU as a
service providers is notnecessarily the kid in their
basement, but it's people thathave access to power.
So former crypto companies orcrypto companies that have
access to power just somebodythat has raw access to power
they are getting funding.
There's a lot of private equitymoney that's out there looking
(23:54):
to go and invest in this space.
So build a data center, buy abunch of NVIDIA gear, set it up
to go run GPU as a service.
We actually at Worldwide areworking with them to be able to
develop those services.
So we have a whole group offolks that work with those
different types of companies asthey want to get these up and
running, to help build them.
But I think that's going to bea real interesting thing over
(24:16):
the next three years becausethis access to power is so
critical and if you look at whatthe power companies are saying,
they can't generate enough,cannot meet the demands of what
some of the hyperscalers areasking them for, even in an RFP
instance.
So you have these smallerplaces that have access to this
power and they're going to godevelop it.
(24:37):
So I think we see anywhere from50 to 100 of these that are
sort of out there in earlystages in some way.
I think they'll probably likemost things that are kind of a
little bit like the Wild West.
You'll see a handful of themactually survive and really get
up and running beyond what theydo.
But we're very pleased with theones we've been working with to
(24:59):
help them.
I think that as we see themdevelop and we see the demand
for GPU as a service, it is alittle bit like a build it and
they will come.
People will end up coming andusing it.
I think the dependencies aregoing to be on how sensitive
that data is, what type ofmarket they're in.
I would see that much like acar, where folks will buy based
(25:21):
on what their requirements are.
You might have folks that gofor a more fully managed, more
luxury experience, where theyhave somebody with a lot of
enterprise experience to getthat going.
Or you might see moremid-market companies that can't
afford that, using some of theseother providers, or even
verticalization of thoseproviders, as maybe somebody
(25:41):
focuses on utilities or somebodywho's focusing on retail and
you'll see them break into thosetypes of segments, but we're in
the very early stages of this.
There's a handful of providersthat we partner with that Don
and I have a portfolio that wehave and that we believe are
good partners for our clients,but there's a lot more to come.
Speaker 3 (26:00):
Yeah so you go ahead, don.
So I'd like to comment on that alittle bit is these are the
kinds of workloads that we'retalking about are very, very
different than a traditional itworkload.
Um, uh, supercomputing, which isreally what we're talking about
(26:23):
here, and it computing are verydifferent specialties.
Uh, from an operational pointof view and this is where I
think there's a lot of theorganizations that are either
looking at doing it themselves,or some of our customers doing
it themselves, or many of thesmall new entrants who have
(26:44):
access to power or trying tostand up environments, don't
quite realize just how hard itis to run high-performance
computing systems on day two andday 100 and day 200 type
operations.
So these things are big,they're complex, they consume a
(27:06):
lot of power, they haverequirements that have never
been delivered or extremelyrarely delivered inside
corporate data centers.
Partners is evaluate them andsee whether you, whether they're
(27:34):
even in a position to run thesekinds of architectures and run
these systems, um, in aresponsible way to the business.
Um, I think that's an importantthing to talk about, especially
dealing with some of the large,highly experienced GPU as a
service providers.
It's just how hard it is to runthese environments.
Speaker 1 (27:51):
Expand on that a little bit.
What do you mean by in aresponsible way to the business?
Speaker 3 (27:56):
Failure rates.
When you, when you pump a 130kilowatts into a very small
space you are going to, you aregoing to generate more failures
in the electronics.
It's just.
I mean, that's just, that'sjust physics.
And so you have to do thingslike how do you manage spares,
(28:17):
these things?
How do you manage extremelyexpensive spares for things like
GPUs?
How do you, how do you buildresiliency into the system that
is right for the business?
High performance, traditionalhigh performance computing
environments, kinds of thingsyou see in national laboratories
(28:38):
and three-letter agencies andplaces like that.
They don't have the same levelof redundancy built into them.
They get their resiliencythrough different methods.
They go through a method ofcheckpointing, which is not what
you do in a traditional ITspace.
(29:00):
You have to understand howthose workflows go through the
system and understand sort ofthem from an operational level,
how to deal with failure, how todeal with, you know, making
sure that the that your very,very expensive resources, um,
are properly utilized, um andhow to account for that Um again
(29:24):
, different kinds of things thanyou would get in a traditional
it space.
Uh, having those kinds ofskills sort of built into the
organization is not somethingyou can just turn on overnight.
So these are some of the morenuanced and day two and day 100
(29:47):
kind of conversations that we'rehaving with customers around
sort of how, how, how to makesure you're getting the value to
the business that that you'repaying for.
Speaker 1 (30:00):
Yeah and Don, one of the funny things that you know I
enjoy hearing I've heard yousay on multiple occasions is you
know how you might findyourself in a meeting and you're
just delivering only bad news.
So if you're talking about thatday to day 100, talk to me a
little bit more about what youmean by that bearer of bad news.
And is it just that the realityof the matter is that not many
are ready for this?
Speaker 3 (30:21):
Well, what, what are, what are I mean I?
I mean I say that in jest, youknow around that, but really
what our customers want is thattrustworthy, informed opinions
about how to get to the businessoutcome they're trying to get
to.
And quite often part of our jobis to look at that customer and
(30:46):
look at their environment andsay your environment is not
going to work or youroperational model is not going
to work, and we're going toexplain why.
As opposed to, I mean, givingbad news is easy, giving an
informed opinion is what ourcustomers are asking us for Now.
(31:08):
Again, we have that experienceright.
I have plenty of time doingsupercomputing in previous roles
.
I understand sort of what therequirements are at the highest
levels.
I have plenty of experience indealing with traditional IT
(31:28):
environments and the kinds ofthings that you have to do in
order to make those operate froman IT point of view.
Plenty of experience andunderstanding sort of the
capabilities of data centers.
Again, now we're looking at theworkloads that we're getting
from our OEMs like NVIDIA andothers that are going into those
(31:51):
environments.
And again, because WWT has thatexperience in high-performance
computing in IT, you know we canlook at that and we can
honestly go to a customer andsay you're not ready for this,
and here's why Customers lovethat kind of advice.
Telling them everything isgoing to be OK is quite often
(32:13):
not what they need to hear.
Okay is quite often not whatthey need to hear.
And so long as we can tell them, you know that.
You know there's a.
We've recognized what yourproblem is and here's a path
forward.
Speaker 1 (32:34):
That, I think, is the real value that WWT brings to
this.
It brings to our customers inthis equation.
Yeah, chris, from what you'reseeing, do you find
organizations try to pigeonholethemselves in any one of those
particular areas, either on-premor in the cloud, just based on
how they've been doing thingsfrom a traditional standpoint,
or are they open to thoseconversations?
Speaker 2 (32:50):
Yeah, I think they have to be open to them, just
because of the cost involved andyou're not talking about small
investments that they're makingand so they might have some type
of use and been burned by cloudas an example.
I think everybody moving overto cloud had unanticipated costs
that they ended up building up,and so they're trying to
(33:12):
determine, well, with these twoworkloads that are different, is
that cloud cost going to end upbeing something that they don't
want to repeat again either?
But then you know, ultimately,as we're trying to look at and
talk to what they do with theseworkloads, we want to help them
evaluate what's best for them aswell.
So if they have the ability intheir data center to go do it
(33:32):
on-prem and they have money andthe workload requires it, I
think that's something that theywould go down and do, that.
We have some existing folksthat have that capability today
and that's where they'recomfortable doing it and they
prefer to keep it on-prem.
I think what we're seeing alsois and I talked about this a
minute ago is this is still veryearly stage, so where's the
(33:53):
long-term investment going tosit?
And I think the answer is it'sgoing to be a hybrid approach.
So they might keep some of iton-prem, they might run some of
it in the cloud, they might runsome of it in a GPU as a service
or a private hosting or AI as aservice type model, and I think
that's, you know, from a coststandpoint they're going to
evaluate.
We are working on those typesof cost models ourselves so we
(34:14):
can help them better understandhow to make those decisions and
where they fit, so they can be alittle more predictable in the
cost.
I think, if you talk about talkto Gartner or anybody else, I
think that some of theirreporting says that in the next
four or five years that 50% ofthese projects are going to end
up being more expensive thanthey anticipated and go over
(34:34):
budget.
Then they also talk aboutoperationally how about in the
next three or four years?
You know 30 to 40% of these aregoing to fail because they
aren't architected the right wayor the model they chose wasn't
good enough.
So I think there's a level ofpredictability that they're
looking at that we're trying tohelp them with.
But you know we're looking atwhere these want to fit.
Speaker 1 (35:04):
Cost becomes kind of the king when it comes to this
as well, as does it actuallysupport the workload in the way
they want it to work?
Yeah, don, what about?
I mean it's interesting thatyou talk about cost, chris.
I mean certainly everythingkind of boils down to that.
But, don, what does events orinflection points like deep seek
?
How does that change thediscourse of this conversation,
thinking that there's probablygoing to be more deep seek, deep
seek type examples movingforward.
(35:24):
What does that do to that kindof spreading of AI workloads?
Speaker 3 (35:30):
So deep seek.
I mean the industry overrotated a little bit on that,
but there is a message therethat matters, which is this is
an incredibly complicated andcomplex computing problem and in
computer science the way yousolve incredibly complicated,
complex problems to start withis brute force, where you use
(35:53):
brute force algorithms and youand you do, and you start with
that.
What we're seeing withalgorithms and improvements like
deep seek and there there willbe others is we will see good
computer science come in and wewill see improvements in those
algorithms and quite often youcan get very large speed ups and
(36:16):
very large efficiency gains byjust simply using better
algorithms.
And this is for me as an in inmy background as a, as a
computer scientist.
That's where I think there'sgoing to be a lot of interest in
this space.
(36:47):
Now, invariably in those things,there's going to be trade-offs
in those algorithms, there'sgoing to be improvements,
there's going to be blind alleysthat the industry looks at that
.
Some really, really smartpeople are starting to get
involved and looking at it froman algorithm design point of
view, which I think is againvery interesting from a computer
science point of view.
So I'm actually reallyencouraged by it.
(37:10):
I think it's actually a good.
I think it's a sign that theindustry is maturing a little
bit, that we're starting to seethese kinds of incremental
advancements in algorithm design.
Speaker 2 (37:23):
Yeah, and I'm going to also add onto that as well.
You know the idea.
When it first came out, I thinkall of our phones kind of blew
up that you know China's gotsomething and it's going to be
really bad for everybody, andvery quickly.
If you started looking at it,you know open source is very
positive and NVIDIA was verypositive about it, and as people
really looked at it, they said,well, hey, this is just a way,
(37:44):
a far more efficient way, to runsome of these workloads.
And I think everybody's lookingfor some type of increased
efficiency.
And you know there's a term thatyou know I've seen batted
around a lot, which is Javon'sparadox.
And if you look at Javon'sparadox, they kind of talk about
increased efficiency can leadto increased consumption rather
(38:04):
than decreased consumption, andso this started with coal.
When they did it, it wassemiconductors, it was cloud, I
mean.
You see all these examples ofthis.
So I think there's nothing tofear here and I think, in fact,
it'll end up being moreefficient for us to run in the
long run, as we're all learninghow to do this, and I think you
can only look for people thatcontinue to try to innovate and
(38:28):
make this better and I thinkit'll just ultimately lead to
more increased consumption, aswe talked about.
So I think it's an interestingtime.
I think we're only at thebeginning of how people will
innovate, trying to make thiseither more cost effective, more
efficient.
You know you can't justcontinue to have runaway costs
in this space.
I think it's got to be broughtout of reality fairly quickly in
(38:50):
this as well.
Speaker 1 (39:00):
Well, Chris, another thing that you mentioned that I
wanted to touch on, but I will.
I'll flip the question over toDon.
You mentioned how you knowlessons learned from the cloud
period.
You could talk about cloud, youcould talk about virtualization
, you could talk about edge.
What types of lessons learnedfrom those eras, so to speak,
should we be applying to aiworkloads, recognizing the fact
(39:22):
down that early on you saidthese are just different beasts.
Ai workloads are there lessonslearned there?
Speaker 3 (39:28):
I think there are.
I I think one of the I thinkyou didn't mention to me, which
is the, which is the um.
What's the analogy that I givepeople?
10 years ago, everybody wasputting everything onto Hadoop.
I mean, everything was goingonto Hadoop, every single
(39:49):
workload was going to be ananalytics workload.
And, again to me, very similarkinds of decisions were being
made.
We were organizations weresetting up massive, huge, very
expensive, um, um analyticsenvironments, uh, and with the
(40:10):
intention that they'd be movingtons and tons of the, if not the
vast majority of theirworkloads whatever that happened
to be into analyticsenvironments.
There's an immense amount ofvalue in doing analytics, the
way that that's done in thoseenvironments.
(40:32):
But that mania died out.
Sort of cooler heads ended upprevailing and there's
incredible amounts of valuecoming out of the.
You know, coming out of thatwork.
We're in my mind, we're we'resort of in a similar sort of
stage right now is everything.
You know, everything needs tobe tagged and has a tagline
(40:53):
around AI and how it's innate.
Everything is an AI workloadand there will be a lot of
workloads that do benefit frominferencing and from learning
models and RAG models and allthe things that we're talking
about in AI.
But at some point Chris isright there's going to have to
(41:16):
be real value generated fromthis and the people who are
paying for it are going to haveto be able to see that return on
investment.
And we need, as an industry, weneed to get to that place as
quickly as possible, becausethat's when this particular
industry will really start tomature.
Speaker 1 (41:38):
That's when this particular industry will really
start to mature.
No-transcript.
Speaker 2 (42:08):
You know, I think that some of the things that we
everybody's in reaction mode andso there's this idea that this
is all new.
We have to figure out how toreact to it, and I was just
actually talking with a vendorgroup and presenting to them as
part of a panel this week, andone of the things I talked about
was trying to think even moreforward.
You have to look at what thenext iteration is going to be
and then also trying to makesense of the chaos that this
(42:30):
presents to some of the clients,because clients are still
trying to figure it out.
We have done a lot of workwithin Worldwide to educate our
clients and help them to betterunderstand it and, I think, for
the partners that are out therereally helping them understand
as well.
The clients are getting hitwith a lot of noise right now
(42:50):
and a lot of FUD, and so whatcan we do and what can these
partners do to be able to helpthem?
To to really, you know, helpthe customers understand the
landscape, make decisions in theright way and, you know, really
help them in some ways to tomaybe not not overreact to
what's happening right now sothey can make smart decisions
(43:11):
about the future.
Speaker 1 (43:13):
Yeah, Don any questions that you're not seeing
.
Asked a lot, but do deserveanswers at some point.
Speaker 3 (43:23):
So I mean I'd pile on to what Chris just said, which
is the questions I would like tosee our customers coming to us
is around strategic planningaround this is around strategic
planning around this.
Chris is exactly right.
We're seeing an incredibleamount of reactive stuff.
I have a project.
I need to go get it solved.
(43:43):
I need to get it solved in thenext few months.
Those are the vast majority ofthe requests that are in the
pipeline right now.
What I would like to see ourcustomers and the industry
asking a little bit more aboutis what does this look like as a
strategic plan?
How are we going to use thistechnology to transform the
(44:04):
business, given that we have touse this technology at a certain
level?
How are we going to change ourorganization, whether that's we
retrofit our existingenvironments, we look for the
right GPU as a service partners,we look for the right
co-locator partners, we look atthe right kind of cloud services
.
I want to see our customerssort of level up the level of
(44:30):
strategic thinking that they'recoming that then the industry
overall is dealing with.
Again, we're in sort of thismania stage right now and I
would personally be much morecomfortable if things were if a
few more cooler heads prevailedat the moment, but that's the
world we're in right now.
Speaker 1 (44:50):
Yeah, a good final note.
Well, to the two of you, thanksfor taking time out of your
busy schedules.
I know you're always divinginto the ATC going on meetings
that may take you out of yourhometown, so thanks again for
joining.
Speaker 2 (45:05):
Yeah, thanks for having us.
Brian, Appreciate it.
Speaker 1 (45:07):
Okay, as we wrap today's conversation, a few
lessons that stand out clearly,lessons that anyone serious
about scaling AI needs to taketo heart.
Anyone serious about scaling AIneeds to take to heart.
First, ai is not just anotherIT project, because you have to
design for it intentionally orrisk discovering that your data
center simply can't support yourambition.
Second, location matters, butso does sensitivity and scale.
(45:30):
The choice between on-prem colo, gpu as a service or public
cloud isn't just aboutconvenience.
It's about understanding yourworkloads, the sensitivity of
your data and the intensity ofyour compute needs, and how
quickly you expect to grow.
And third, infrastructuredecisions are business decisions
.
Investing in ai infrastructureisn't about building shiny new
(45:52):
tech for its own sake.
It's about deliveringtrustworthy, reliable business
outcomes.
Bottom Bottom line.
If you're serious about AI, youcan't treat infrastructure as
an afterthought.
The success, or lack thereof,of your AI strategy depends on
asking the right questionsbefore deployment.
If you liked this episode ofthe AI Proving Ground podcast,
please consider sharing withfriends and colleagues, and
(46:14):
don't forget to subscribe onyour favorite podcast platform
or on WWTcom.
This episode of the AI ProvingGround podcast was co-produced
by Naz Baker, cara Kuhn, mallorySchaffran and Stephanie Hammond
.
Our audio and video engineer isJohn Knobloch, my name is Brian
Felt and we'll see you nexttime.
AI's insatiable hunger for power is reshaping
the global energy map, and fast.
Just last month, theInternational Energy Agency
warned that data centerelectricity use will double by
the year 2030, with AIresponsible for most of that
surge.
Reuters put an even sharperpoint on it by the end of the
decade, one in eight USelectrons could be flowing into
(00:22):
server halls instead of homes orfactories.
One in eight US electrons couldbe flowing into server halls
instead of homes or factories.
Now, that's the landscape.
But our two guests today, ChrisCampbell, a senior director of
AI solutions, and Don Malero, achief technical advisor, both of
which are colleagues of minehere at WWT, are here to talk
about the reality that today'sAI infrastructure decisions
aren't just technical, they'reoperational and they're
(00:44):
financial out their core.
Now you might be asking why doesall of this matter?
Well, the competitive edge in2025 isn't just having AI, it's
running it where it performsbest, at cost and under a risk
profile you're comfortable with.
Get that calculus wrong and thesame model that promised to cut
fraud or discover a new drugnow becomes a drag.
(01:04):
So stick around.
In a few minutes, don willshare a quietly powerful rule of
thumb he uses before anydeployment for whether a
workload belongs on-prem, incolo or in the cloud.
It's simple, it's actionableand it might save you more money
than the GPU's cost in thefirst place.
This is the AI Proving Groundpodcast from Worldwide
Technology.
Let's get to it, chris.
(01:35):
I get the sense from speakingto either clients or hearing
from experts that we have hereat Worldwide Technology, that
consideration of AI workloadsand how they're deployed can
sometimes maybe even often be anafterthought.
Is that an accurate assessment,and what are you seeing from
that perspective?
Speaker 2 (01:54):
Yeah, there's kind of a few things we're seeing is
there was a lot of folks thatwere doing what I'll call stroke
of the pen type purchases ofequipment, and so they would buy
this equipment without reallythinking about how they were
going to support it from aninfrastructure standpoint, power
standpoint, data centerstandpoint.
So now they're looking at itafter the fact and saying, well,
(02:15):
where should this stuffactually go?
And in some cases they may notbe able to support it from their
data center.
We've seen that in quite a fewdifferent people and really from
our standpoint we say, well,let's go ahead and try to plan
ahead.
So we're really encouraging ourclients now to think ahead.
We do a facilities assessmentand this will land in one of a
few places.
(02:36):
But with that planning, I thinkthat'll help them to really be
able to deploy this in ameaningful way without any
slowdown or anything that'sgoing to happen with them to be
able to delay these projects.
Speaker 1 (02:49):
Yeah, don you know, for those that maybe did treat
it a little bit as anafterthought or thinking that it
would just kind of take care ofitself, what are the stumbling
blocks for them if they don'ttake this into consideration
from the get-go?
Speaker 3 (03:04):
Well, I mean in the facilities business, there's the
iron triangle, which is atriangle between power, space
and cooling.
The first stumbling block thatwe see organizations run into is
power.
They realize that, you know,with an IT rack that, on the
(03:29):
high end, is drawing in the 17to 20 kilowatt range, and
they're talking about deployingan AI rack that starts at 50
kilowatts and can conceivably goup to well over, well over 100,
into like the 130, 150 range.
Right now, power becomes thefirst thing that they have to
(03:54):
deal with Very quickly, and it'sa different kind of power.
We're going from 208 volt powerin a traditional data center to
most of the ai systems arerequiring 415 volt power.
So there's not only is it a lotmore power, it's a different
kind of power.
(04:14):
Um, the second piece to thatthat comes along is cooling.
Obviously, uh, the second lawof thermodynamics you know it
still applies.
Uh, the power you put into therack has to be extracted somehow
.
So cooling becomes the nextissue that organizations and
(04:36):
customers start to deal with.
There's lots of different waysto do cooling, and we can talk
about that at some length, butanother piece of it that comes
along very quickly on top ofthat is the space itself.
Traditional raised floor datacenter raised floor environments
simply can't support the weightof modern AI systems about
(05:08):
things that are well over 3,000pounds going into a data center
that might be built for 1,200pound racks.
So we're dealing with things atthe maximum.
So we're dealing withchallenges on power, space and
cooling.
The final challenge that getsput onto this is the cabling and
the amount of interconnect thatthese systems require, and
(05:29):
we've got lots of issues aroundstructured cabling that goes
into these systems.
That is orders of magnitudelarger than you would get in an
IT workload.
So it's power space, coolingcabling, and that's just to get
(05:51):
to day one operation.
So there's lots of challengesall over the map for this.
Speaker 1 (05:57):
Yeah, I like that iron triangle.
Did you come up with that onyour own?
Is that an overpriced area?
Speaker 3 (06:03):
That's a traditional thing in the facilities business
that certainly my colleaguesBruce and Mike have been putting
up in front of customers for along time.
I'm relatively new to thehardcore facilities end of the
business.
I come at it from a compute andan IT operational point of view
(06:28):
and it's been a real educationworking with the facilities
infrastructure team and thedepth of knowledge that they
have.
But yeah, I can't take creditfor that.
I'm going to give full creditto Bruce Gray and Mike Parham on
that, for teaching me aboutfacilities, Sure, sure.
Speaker 1 (06:52):
Well, chris, maybe take a step back for us here and
articulate, just for those thatmay not be familiar, what
exactly AI workload means forour enterprise customers.
What exactly AI workload meansfor our enterprise customers?
Is it different fromtraditional IT workloads that
we've been talking about overthe past, however many decades?
Speaker 2 (07:14):
And if it is different, why is it different?
Yeah, you know this is not yourfather's AI workload or
infrastructure workload thatthey have.
It requires a fully different,as Don alluded to different
setup in the data center,different amounts of power, but
the workload itself really isyou kind of consider what folks
are trying to accomplish.
So they start with a use case,they have some data that they
(07:37):
use to point to, and then theyuse a training model within AI
to be able to go and train it,and that training model requires
a lot of compute power.
That's why NVIDIA is so frontand center in this is that
they've got these chips andallows them to really process
this in a really fast way.
That's why, when you use aGPT-4 or some other kind of
(07:59):
chatbot, it reacts so quicklyand this is why they've got such
a dominant place in theindustry.
But from a workload standpoint,that workload is how are they
processing that informationrapidly to get the answer that
they need to?
And so ultimately, when ittrains, it requires a much
stronger and more powerfulinfrastructure to run on.
(08:20):
But once they can move toinferencing, which means that
they're actually processing on amore long level, and it's
actually what they're doing,from the time they train it
through the rest of its lifespan, is inferencing.
That requires a slightly lesspowerful infrastructure, and you
could do it in a variety ofways, maybe even using different
(08:41):
types of infrastructure for itthat allows it to be able to run
and not require that power.
So it is very different.
This is why we're seeing thisrapid shift in the data center
space to try to find spaces forthese super pods that NVIDIA is
putting out there and otherplaces, because they really
can't fit in any other type oftraditional data center and in
(09:02):
many cases, the legacy datacenters that they have can't
even be retrofitted with therequirements needed to supply it
.
So it's vastly different thanwhat it has been in the past.
Speaker 1 (09:22):
Yeah, and we'll get into talking about where these
workloads can and should run,but Don real quick as well.
Are all AI workloads createdequal?
I'm assuming that there's alevel of nuance to all of this,
and what are some of thoseconsiderations that we should be
thinking about, that ourclients should be thinking about
as they're assessing their AIworkloads and then as we start
(09:44):
to think about where they shouldrun?
Speaker 3 (09:48):
In some way AI workloads are created equal and
in many other ways they're not.
They're all pushing the limit.
They're equal in the sense thatthey are pushing the limits of
what is capable in the industryat the moment, whether that's
(10:11):
leaning on what the computeengine is and quite often NVIDIA
GPUs, the networkinginfrastructure that is going
with that.
There are a lot of very similarconsiderations between
organizations the latency, thelove, you know, the latency, the
bandwidth that happens on thenetwork side and on the storage
side, you know, in making surethat the right data is in the
(10:33):
right place at the right time.
Those are all equally importantconsiderations that
organizations have to look atand that and there are common or
reference solutions to to allof those problems and that's
sort of what we do in the AIPGis is we have those reference
(10:55):
designs, we have them built out,we have good examples of what,
of, of what proper referencearchitecture looks like for
those solutions.
Speaker 1 (11:06):
Yeah, aipg being the AI proving ground that we have
here at WWT to help our clientsgo through their AI journey.
Chris Don mentioned at leastseveral of them computing
storage.
What about data sensitivity,security, latency, things like
that?
What else do we need to beconsidering?
Speaker 2 (11:28):
Yeah, I would say that.
You know, depending upon whatthey're trying to accomplish and
the sensitivity of that data,certainly you have to consider
that and you know that's trulysomething that, as we're looking
and working with some of theearly adopters, they do not want
this data to be any place otherthan an on-prem environment or
a private environment, and so Ithink that as we continue to
(11:48):
progress and more use cases arereleased and they have less
sensitivity about that data,you're going to see other areas,
maybe the cloud or other placesthat'll go, but that data
sensitivity is certainly anissue and then in a location and
I think what we're seeing nowis you could train this in one
area and then ultimately, as yougo out to do the inferencing,
(12:10):
you may want to move it withmore proximity to a cloud source
or other sources so it can runmore effectively and efficiency
and not have that latency.
Speaker 1 (12:20):
And where does considerations of the use case
come in, if at all?
Speaker 2 (12:27):
Well, the use case will depend on how quickly they
want that information in manycases.
So you know what we're seeingin, for example, finance or
we're seeing in health care.
Those are really the earlyadopters that we're seeing.
Life sciences are doing that aswell as some of the retail, and
so that proximity and speed tothat information, depending upon
(12:48):
how quickly they want to get itlet's just say it's fraud
protection that they might beusing within a financial
institution They'll want thatresponse time to be very, very
fast.
And Don's certainly an expertin that area and can jump on and
add to whatever I might bemissing, but I think that's
something they consider as well.
Speaker 1 (13:06):
Yeah, don anything to add there.
Speaker 3 (13:08):
Sure, chris is exactly right.
The underlying infrastructureis important, but the use case
and the business value that goeson top of it is what the
customers are ultimatelyinvesting in.
They're looking at thatbusiness outcome and that's
(13:29):
where all of the other layers ofan AI solution whether that's
learning or whether that'sinferencing or even traditional
AI ML that's what matters to thecustomer.
The infrastructure piece of itis simply a means to an end.
So we have to be we're verycareful about how we build out
(13:53):
those environments whetherthat's a retrofit on premise,
because sometimes it might befor very modest footprints, but
we've seen that some one of thefinancial institutions I'm
working with they're doing.
They're doing theirinferencing-prem because they
can't eke out enough power andcooling in their existing data
(14:14):
center to do it.
For other organizations, they'rebuilding and retrofitting
dedicated environment relativelysmall compared to their overall
footprint, but still dedicatedand reasonably significant
environments in their own datacenters.
Or we're working with customerswho are placing workloads in
(14:38):
co-locators, and some of our keyco-locator partners there are
adapting their environments forvery high scale, very high load
AI type deployments.
And finally, we're looking in,as well as the cloud, the, the
the old cloud service providers,we're looking at new ones that
(15:02):
are specifically dedicated toproviding GPU and high
performance computing in asecure environment for
organizations that have thathave those requirements.
Chris is exactly right.
These workloads are.
They are core to the businessesthat we're dealing with.
(15:24):
They are quite often some ofthe most important workflows and
applications that they'rerunning.
You can, fraud detection is acritical workload at a bank, and
so deploying that and doing itcorrectly and doing it
responsibly is absolutelycritical for these organizations
(15:44):
, and they're having a really,really hard time doing it on
premise with modern computingequipment.
Speaker 1 (15:50):
Really really hard time doing it on premise with
modern computing equipment.
Well, chris, we mentioned thoseoptions there Colo, cloud,
on-prem.
What are you seeing in themarket right now in terms of the
most appropriate way to divvyall that up?
Speaker 2 (16:03):
So the answer is it depends.
I think if you, right now,there's three options that
people are really looking at,and if you look at what some of
the analysts say, and they'retalking about a third of them
being on-prem, a third of thembeing in the cloud and a third
of them being in some kind ofprivate hosted environment like
an AI as a service or a GPU as aservice, which is a leased GPU
(16:28):
environment, and ideally, Ithink that middle area, that
last area I talked about, whichis the GPU as a service, private
hosting is going to be far moreprominent.
The challenges that they faceare if you want to go on-prem,
your data center may not beready.
And so, to Don's point, some ofthese folks might be eking out
inferencing that they have, andthey might be doing that to do
(16:48):
the best they can.
In other cases, they're saying,well, maybe we can try to
retrofit it, but pushing it tothe cloud or pushing it to a
hosting provider like an Equinixor a digital realty, I think,
makes a little bit more sense insome of these cases.
Now, you're not going to go putin the cloud if you've got some
kind of sensitivity to the data.
So that idea that this privatehosting environment is going to
(17:10):
be something that's prominentfor some of these workloads, I
think is very true.
But as these early adopters comeon, some of the easy button is
well, we're going to just go anddo this in the cloud We've
already got an AWS account orthey're going to go ahead and
buy something and put it in aprivate hosted environment
because they're going to make alonger term commitment.
But the GPU as a serviceproviders that I mentioned is
(17:32):
something where, if you want ashort term testing and we're
seeing this with our partnerslike CoreWeave and Scott Data
Center, applied Digital they allare getting a lot of business
right now and people just usingshort term testing so they can
prove out some type of use casethat they're working on or be
able to go get fundings, muchlike they're doing in our AIPG,
(17:52):
but with their own data.
So it's a really interestingtime right now to see this.
But as this business continuesto build over time, I think
you're going to see thesedecisions being made and while
folks will invest in HPA,on-prem, high-performance
architecture on prem, I do thinkthat some of the easy buttons
for them, instead of building adata center, buying $20 million
(18:14):
worth of gear is to figure out aplace to go and host it or to
lease it from someplace.
It's kind of that build versusbuy discussion that people have.
Speaker 1 (18:23):
Yeah, Don.
What do you make of that GPU asa service angle, and doesn't
that also just run the risk ofpotentially running up the cost?
If people are looking to youknow, take advantage of that
scarcity.
Speaker 3 (18:36):
So.
So there's there's as with allthese solutions, there's a I
mean, there's, there's a lot ofnuance in the actual solution.
So, even even when people talkabout GPU as a service, you'll
have environments.
You'll be able to go to a GPUas a service provider, and
(18:59):
there's lots of them that we'repartnering with Scott Data
Center, coreweave, lambda Labs,Applied Digital there are lots
of them in this space.
They all have slightlydifferent product offerings and
dealing with them and gettingyour workload working there is
(19:19):
non-trivial.
For example, they may have adebit.
You may be in a situation whereyou require a cage, dedicated
environment that even the GPU asa service provider doesn't have
access to, because you want torun confidential workloads in
there.
That is some of the offeringsfrom this.
(19:40):
There's much more traditional,even all the way down to
virtualized environments whereyou're multi-tenanting on
hardware with other corporationsor other clients of those
organizations.
So even the offerings thatyou'll see from a GPU as a
(20:00):
service provider, there's a lotof nuance there.
There's also a lot of nuanceabout the level of service
you're going to get from them,which is very different than
what you would get in a cloudservice provider.
Cloud service provider is goingto provide you a complete
end-to-end solution wherethey're going to manage all of
the infrastructure, all of theservices and everything that
goes with that In a GPU.
(20:23):
As a service provider, theymight just be providing you the
bare metal, or they might beproviding you up to the
operating system level, or theymight be providing you up to the
operating system level, or theymight be providing services up
to the Kubernetes and Slurmlevel, or even beyond that.
And this is why I think there'sa lot of value in a customer
coming to WWT, partnering withboth WWT and the GPU as a
(20:48):
service provider to provide anend-to-end solution that
includes access to the hardware,access to a high-performance
computing environment that hasthe right services all the way
up through the application.
And this is where it's a realcase of one plus one equals a
(21:09):
lot more than two for thecustomer.
Because that's where the valueof the partnership is, between
WWT and our GPU as a serviceprovider partners to the
customers.
I think it's not only is itjust a matter of having a place
to run your workloads, it'sactually, you know, we really
(21:33):
believe that you can execute abetter solution for the customer
and those kinds of environments.
Speaker 2 (21:39):
Yeah, brian, I'm going to plus one what Don was
talking about here, but reallytalk about.
You know the investment peoplehave to make.
So if you're asking, if you're aCIO or you're running a center
of excellence at a client andyou're trying to go ask for $5,
$10, $15 million to go to a usecase, they are sometimes wanting
to prove that out before theyget approval from their board on
(22:00):
that cost.
So, using a GPU as a serviceprovider that already has that
equipment in place we see thisagain in our AIPG as well, in
our AI proving ground, wherefolks will come in and just want
to run it on one of the 15reference architectures that we
have.
Well, that's kind of a firststep.
And then, when you really wantto use your own data rather than
some kind of synthetic datajust we use in the AIPG, that's
(22:23):
where they could go into a GPUas a service provider, really
prove this out and determinewhether or not it works, and I
think that's something thatsaves them a lot of time.
It probably saves them somemoney in the initial stages and
so you know, we see a reallyhigh level of use coming for
this GPU as a service as folkscontinue to put their budgets
toward AI projects.
Speaker 3 (22:45):
This episode is supported by Equinix.
Equinix connects businessesglobally through its extensive
network of data centers,enabling seamless digital
interactions.
Expand your reach withEquinix's interconnected
infrastructure.
Speaker 1 (23:06):
What's the market look like?
I heard a story the other daytalking about how there's people
out there looking to build anetwork of folks that just have
excess capacity of their GPUs,thinking like kids in their
parents' basement playing on aGPU card in a video game.
Obviously that's kind of an offangle of that area, but is that
(23:27):
market going to continue toexpand and grow or is it going
to be tight?
Speaker 2 (23:32):
area.
But is that market going tocontinue to expand and grow or
is it going to be tight?
Yeah, the biggest thing thatwe're seeing for GPU as a
service providers is notnecessarily the kid in their
basement, but it's people thathave access to power.
So former crypto companies orcrypto companies that have
access to power just somebodythat has raw access to power
they are getting funding.
There's a lot of private equitymoney that's out there looking
(23:54):
to go and invest in this space.
So build a data center, buy abunch of NVIDIA gear, set it up
to go run GPU as a service.
We actually at Worldwide areworking with them to be able to
develop those services.
So we have a whole group offolks that work with those
different types of companies asthey want to get these up and
running, to help build them.
But I think that's going to bea real interesting thing over
(24:16):
the next three years becausethis access to power is so
critical and if you look at whatthe power companies are saying,
they can't generate enough,cannot meet the demands of what
some of the hyperscalers areasking them for, even in an RFP
instance.
So you have these smallerplaces that have access to this
power and they're going to godevelop it.
(24:37):
So I think we see anywhere from50 to 100 of these that are
sort of out there in earlystages in some way.
I think they'll probably likemost things that are kind of a
little bit like the Wild West.
You'll see a handful of themactually survive and really get
up and running beyond what theydo.
But we're very pleased with theones we've been working with to
(24:59):
help them.
I think that as we see themdevelop and we see the demand
for GPU as a service, it is alittle bit like a build it and
they will come.
People will end up coming andusing it.
I think the dependencies aregoing to be on how sensitive
that data is, what type ofmarket they're in.
I would see that much like acar, where folks will buy based
(25:21):
on what their requirements are.
You might have folks that gofor a more fully managed, more
luxury experience, where theyhave somebody with a lot of
enterprise experience to getthat going.
Or you might see moremid-market companies that can't
afford that, using some of theseother providers, or even
verticalization of thoseproviders, as maybe somebody
(25:41):
focuses on utilities or somebodywho's focusing on retail and
you'll see them break into thosetypes of segments, but we're in
the very early stages of this.
There's a handful of providersthat we partner with that Don
and I have a portfolio that wehave and that we believe are
good partners for our clients,but there's a lot more to come.
Speaker 3 (26:00):
Yeah so you go ahead, don.
So I'd like to comment on that alittle bit is these are the
kinds of workloads that we'retalking about are very, very
different than a traditional itworkload.
Um, uh, supercomputing, which isreally what we're talking about
(26:23):
here, and it computing are verydifferent specialties.
Uh, from an operational pointof view and this is where I
think there's a lot of theorganizations that are either
looking at doing it themselves,or some of our customers doing
it themselves, or many of thesmall new entrants who have
(26:44):
access to power or trying tostand up environments, don't
quite realize just how hard itis to run high-performance
computing systems on day two andday 100 and day 200 type
operations.
So these things are big,they're complex, they consume a
(27:06):
lot of power, they haverequirements that have never
been delivered or extremelyrarely delivered inside
corporate data centers.
Partners is evaluate them andsee whether you, whether they're
(27:34):
even in a position to run thesekinds of architectures and run
these systems, um, in aresponsible way to the business.
Um, I think that's an importantthing to talk about, especially
dealing with some of the large,highly experienced GPU as a
service providers.
It's just how hard it is to runthese environments.
Speaker 1 (27:51):
Expand on that a little bit.
What do you mean by in aresponsible way to the business?
Speaker 3 (27:56):
Failure rates.
When you, when you pump a 130kilowatts into a very small
space you are going to, you aregoing to generate more failures
in the electronics.
It's just.
I mean, that's just, that'sjust physics.
And so you have to do thingslike how do you manage spares,
(28:17):
these things?
How do you manage extremelyexpensive spares for things like
GPUs?
How do you, how do you buildresiliency into the system that
is right for the business?
High performance, traditionalhigh performance computing
environments, kinds of thingsyou see in national laboratories
(28:38):
and three-letter agencies andplaces like that.
They don't have the same levelof redundancy built into them.
They get their resiliencythrough different methods.
They go through a method ofcheckpointing, which is not what
you do in a traditional ITspace.
(29:00):
You have to understand howthose workflows go through the
system and understand sort ofthem from an operational level,
how to deal with failure, how todeal with, you know, making
sure that the that your very,very expensive resources, um,
are properly utilized, um andhow to account for that Um again
(29:24):
, different kinds of things thanyou would get in a traditional
it space.
Uh, having those kinds ofskills sort of built into the
organization is not somethingyou can just turn on overnight.
So these are some of the morenuanced and day two and day 100
(29:47):
kind of conversations that we'rehaving with customers around
sort of how, how, how to makesure you're getting the value to
the business that that you'repaying for.
Speaker 1 (30:00):
Yeah and Don, one of the funny things that you know I
enjoy hearing I've heard yousay on multiple occasions is you
know how you might findyourself in a meeting and you're
just delivering only bad news.
So if you're talking about thatday to day 100, talk to me a
little bit more about what youmean by that bearer of bad news.
And is it just that the realityof the matter is that not many
are ready for this?
Speaker 3 (30:21):
Well, what, what are, what are I mean I?
I mean I say that in jest, youknow around that, but really
what our customers want is thattrustworthy, informed opinions
about how to get to the businessoutcome they're trying to get
to.
And quite often part of our jobis to look at that customer and
(30:46):
look at their environment andsay your environment is not
going to work or youroperational model is not going
to work, and we're going toexplain why.
As opposed to, I mean, givingbad news is easy, giving an
informed opinion is what ourcustomers are asking us for Now.
(31:08):
Again, we have that experienceright.
I have plenty of time doingsupercomputing in previous roles
.
I understand sort of what therequirements are at the highest
levels.
I have plenty of experience indealing with traditional IT
(31:28):
environments and the kinds ofthings that you have to do in
order to make those operate froman IT point of view.
Plenty of experience andunderstanding sort of the
capabilities of data centers.
Again, now we're looking at theworkloads that we're getting
from our OEMs like NVIDIA andothers that are going into those
(31:51):
environments.
And again, because WWT has thatexperience in high-performance
computing in IT, you know we canlook at that and we can
honestly go to a customer andsay you're not ready for this,
and here's why Customers lovethat kind of advice.
Telling them everything isgoing to be OK is quite often
(32:13):
not what they need to hear.
Okay is quite often not whatthey need to hear.
And so long as we can tell them, you know that.
You know there's a.
We've recognized what yourproblem is and here's a path
forward.
Speaker 1 (32:34):
That, I think, is the real value that WWT brings to
this.
It brings to our customers inthis equation.
Yeah, chris, from what you'reseeing, do you find
organizations try to pigeonholethemselves in any one of those
particular areas, either on-premor in the cloud, just based on
how they've been doing thingsfrom a traditional standpoint,
or are they open to thoseconversations?
Speaker 2 (32:50):
Yeah, I think they have to be open to them, just
because of the cost involved andyou're not talking about small
investments that they're makingand so they might have some type
of use and been burned by cloudas an example.
I think everybody moving overto cloud had unanticipated costs
that they ended up building up,and so they're trying to
(33:12):
determine, well, with these twoworkloads that are different, is
that cloud cost going to end upbeing something that they don't
want to repeat again either?
But then you know, ultimately,as we're trying to look at and
talk to what they do with theseworkloads, we want to help them
evaluate what's best for them aswell.
So if they have the ability intheir data center to go do it
(33:32):
on-prem and they have money andthe workload requires it, I
think that's something that theywould go down and do, that.
We have some existing folksthat have that capability today
and that's where they'recomfortable doing it and they
prefer to keep it on-prem.
I think what we're seeing alsois and I talked about this a
minute ago is this is still veryearly stage, so where's the
(33:53):
long-term investment going tosit?
And I think the answer is it'sgoing to be a hybrid approach.
So they might keep some of iton-prem, they might run some of
it in the cloud, they might runsome of it in a GPU as a service
or a private hosting or AI as aservice type model, and I think
that's, you know, from a coststandpoint they're going to
evaluate.
We are working on those typesof cost models ourselves so we
(34:14):
can help them better understandhow to make those decisions and
where they fit, so they can be alittle more predictable in the
cost.
I think, if you talk about talkto Gartner or anybody else, I
think that some of theirreporting says that in the next
four or five years that 50% ofthese projects are going to end
up being more expensive thanthey anticipated and go over
(34:34):
budget.
Then they also talk aboutoperationally how about in the
next three or four years?
You know 30 to 40% of these aregoing to fail because they
aren't architected the right wayor the model they chose wasn't
good enough.
So I think there's a level ofpredictability that they're
looking at that we're trying tohelp them with.
But you know we're looking atwhere these want to fit.
Speaker 1 (35:04):
Cost becomes kind of the king when it comes to this
as well, as does it actuallysupport the workload in the way
they want it to work?
Yeah, don, what about?
I mean it's interesting thatyou talk about cost, chris.
I mean certainly everythingkind of boils down to that.
But, don, what does events orinflection points like deep seek
?
How does that change thediscourse of this conversation,
thinking that there's probablygoing to be more deep seek, deep
seek type examples movingforward.
(35:24):
What does that do to that kindof spreading of AI workloads?
Speaker 3 (35:30):
So deep seek.
I mean the industry overrotated a little bit on that,
but there is a message therethat matters, which is this is
an incredibly complicated andcomplex computing problem and in
computer science the way yousolve incredibly complicated,
complex problems to start withis brute force, where you use
(35:53):
brute force algorithms and youand you do, and you start with
that.
What we're seeing withalgorithms and improvements like
deep seek and there there willbe others is we will see good
computer science come in and wewill see improvements in those
algorithms and quite often youcan get very large speed ups and
(36:16):
very large efficiency gains byjust simply using better
algorithms.
And this is for me as an in inmy background as a, as a
computer scientist.
That's where I think there'sgoing to be a lot of interest in
this space.
(36:47):
Now, invariably in those things,there's going to be trade-offs
in those algorithms, there'sgoing to be improvements,
there's going to be blind alleysthat the industry looks at that
.
Some really, really smartpeople are starting to get
involved and looking at it froman algorithm design point of
view, which I think is againvery interesting from a computer
science point of view.
So I'm actually reallyencouraged by it.
(37:10):
I think it's actually a good.
I think it's a sign that theindustry is maturing a little
bit, that we're starting to seethese kinds of incremental
advancements in algorithm design.
Speaker 2 (37:23):
Yeah, and I'm going to also add onto that as well.
You know the idea.
When it first came out, I thinkall of our phones kind of blew
up that you know China's gotsomething and it's going to be
really bad for everybody, andvery quickly.
If you started looking at it,you know open source is very
positive and NVIDIA was verypositive about it, and as people
really looked at it, they said,well, hey, this is just a way,
(37:44):
a far more efficient way, to runsome of these workloads.
And I think everybody's lookingfor some type of increased
efficiency.
And you know there's a term thatyou know I've seen batted
around a lot, which is Javon'sparadox.
And if you look at Javon'sparadox, they kind of talk about
increased efficiency can leadto increased consumption rather
(38:04):
than decreased consumption, andso this started with coal.
When they did it, it wassemiconductors, it was cloud, I
mean.
You see all these examples ofthis.
So I think there's nothing tofear here and I think, in fact,
it'll end up being moreefficient for us to run in the
long run, as we're all learninghow to do this, and I think you
can only look for people thatcontinue to try to innovate and
(38:28):
make this better and I thinkit'll just ultimately lead to
more increased consumption, aswe talked about.
So I think it's an interestingtime.
I think we're only at thebeginning of how people will
innovate, trying to make thiseither more cost effective, more
efficient.
You know you can't justcontinue to have runaway costs
in this space.
I think it's got to be broughtout of reality fairly quickly in
(38:50):
this as well.
Speaker 1 (39:00):
Well, Chris, another thing that you mentioned that I
wanted to touch on, but I will.
I'll flip the question over toDon.
You mentioned how you knowlessons learned from the cloud
period.
You could talk about cloud, youcould talk about virtualization
, you could talk about edge.
What types of lessons learnedfrom those eras, so to speak,
should we be applying to aiworkloads, recognizing the fact
(39:22):
down that early on you saidthese are just different beasts.
Ai workloads are there lessonslearned there?
Speaker 3 (39:28):
I think there are.
I I think one of the I thinkyou didn't mention to me, which
is the, which is the um.
What's the analogy that I givepeople?
10 years ago, everybody wasputting everything onto Hadoop.
I mean, everything was goingonto Hadoop, every single
(39:49):
workload was going to be ananalytics workload.
And, again to me, very similarkinds of decisions were being
made.
We were organizations weresetting up massive, huge, very
expensive, um, um analyticsenvironments, uh, and with the
(40:10):
intention that they'd be movingtons and tons of the, if not the
vast majority of theirworkloads whatever that happened
to be into analyticsenvironments.
There's an immense amount ofvalue in doing analytics, the
way that that's done in thoseenvironments.
(40:32):
But that mania died out.
Sort of cooler heads ended upprevailing and there's
incredible amounts of valuecoming out of the.
You know, coming out of thatwork.
We're in my mind, we're we'resort of in a similar sort of
stage right now is everything.
You know, everything needs tobe tagged and has a tagline
(40:53):
around AI and how it's innate.
Everything is an AI workloadand there will be a lot of
workloads that do benefit frominferencing and from learning
models and RAG models and allthe things that we're talking
about in AI.
But at some point Chris isright there's going to have to
(41:16):
be real value generated fromthis and the people who are
paying for it are going to haveto be able to see that return on
investment.
And we need, as an industry, weneed to get to that place as
quickly as possible, becausethat's when this particular
industry will really start tomature.
Speaker 1 (41:38):
That's when this particular industry will really
start to mature.
No-transcript.
Speaker 2 (42:08):
You know, I think that some of the things that we
everybody's in reaction mode andso there's this idea that this
is all new.
We have to figure out how toreact to it, and I was just
actually talking with a vendorgroup and presenting to them as
part of a panel this week, andone of the things I talked about
was trying to think even moreforward.
You have to look at what thenext iteration is going to be
and then also trying to makesense of the chaos that this
(42:30):
presents to some of the clients,because clients are still
trying to figure it out.
We have done a lot of workwithin Worldwide to educate our
clients and help them to betterunderstand it and, I think, for
the partners that are out therereally helping them understand
as well.
The clients are getting hitwith a lot of noise right now
(42:50):
and a lot of FUD, and so whatcan we do and what can these
partners do to be able to helpthem?
To to really, you know, helpthe customers understand the
landscape, make decisions in theright way and, you know, really
help them in some ways to tomaybe not not overreact to
what's happening right now sothey can make smart decisions
(43:11):
about the future.
Speaker 1 (43:13):
Yeah, Don any questions that you're not seeing
.
Asked a lot, but do deserveanswers at some point.
Speaker 3 (43:23):
So I mean I'd pile on to what Chris just said, which
is the questions I would like tosee our customers coming to us
is around strategic planningaround this is around strategic
planning around this.
Chris is exactly right.
We're seeing an incredibleamount of reactive stuff.
I have a project.
I need to go get it solved.
(43:43):
I need to get it solved in thenext few months.
Those are the vast majority ofthe requests that are in the
pipeline right now.
What I would like to see ourcustomers and the industry
asking a little bit more aboutis what does this look like as a
strategic plan?
How are we going to use thistechnology to transform the
(44:04):
business, given that we have touse this technology at a certain
level?
How are we going to change ourorganization, whether that's we
retrofit our existingenvironments, we look for the
right GPU as a service partners,we look for the right
co-locator partners, we look atthe right kind of cloud services
.
I want to see our customerssort of level up the level of
(44:30):
strategic thinking that they'recoming that then the industry
overall is dealing with.
Again, we're in sort of thismania stage right now and I
would personally be much morecomfortable if things were if a
few more cooler heads prevailedat the moment, but that's the
world we're in right now.
Speaker 1 (44:50):
Yeah, a good final note.
Well, to the two of you, thanksfor taking time out of your
busy schedules.
I know you're always divinginto the ATC going on meetings
that may take you out of yourhometown, so thanks again for
joining.
Speaker 2 (45:05):
Yeah, thanks for having us.
Brian, Appreciate it.
Speaker 1 (45:07):
Okay, as we wrap today's conversation, a few
lessons that stand out clearly,lessons that anyone serious
about scaling AI needs to taketo heart.
Anyone serious about scaling AIneeds to take to heart.
First, ai is not just anotherIT project, because you have to
design for it intentionally orrisk discovering that your data
center simply can't support yourambition.
Second, location matters, butso does sensitivity and scale.
(45:30):
The choice between on-prem colo, gpu as a service or public
cloud isn't just aboutconvenience.
It's about understanding yourworkloads, the sensitivity of
your data and the intensity ofyour compute needs, and how
quickly you expect to grow.
And third, infrastructuredecisions are business decisions
.
Investing in ai infrastructureisn't about building shiny new
(45:52):
tech for its own sake.
It's about deliveringtrustworthy, reliable business
outcomes.
Bottom Bottom line.
If you're serious about AI, youcan't treat infrastructure as
an afterthought.
The success, or lack thereof,of your AI strategy depends on
asking the right questionsbefore deployment.
If you liked this episode ofthe AI Proving Ground podcast,
please consider sharing withfriends and colleagues, and
(46:14):
don't forget to subscribe onyour favorite podcast platform
or on WWTcom.
This episode of the AI ProvingGround podcast was co-produced
by Naz Baker, cara Kuhn, mallorySchaffran and Stephanie Hammond
.
Our audio and video engineer isJohn Knobloch, my name is Brian
Felt and we'll see you nexttime.
Popular Podcasts
Crime Junkie
Does hearing about a true crime case always leave you scouring the internet for the truth behind the story? Dive into your next mystery with Crime Junkie. Every Monday, join your host Ashley Flowers as she unravels all the details of infamous and underreported true crime cases with her best friend Brit Prawat. From cold cases to missing persons and heroes in our community who seek justice, Crime Junkie is your destination for theories and stories you won’t hear anywhere else. Whether you're a seasoned true crime enthusiast or new to the genre, you'll find yourself on the edge of your seat awaiting a new episode every Monday. If you can never get enough true crime... Congratulations, you’ve found your people. Follow to join a community of Crime Junkies! Crime Junkie is presented by audiochuck Media Company.
24/7 News: The Latest
The latest news in 4 minutes updated every hour, every day.
Stuff You Should Know
If you've ever wanted to know about champagne, satanism, the Stonewall Uprising, chaos theory, LSD, El Nino, true crime and Rosa Parks, then look no further. Josh and Chuck have you covered.