October 22, 2024 • 34 mins
According to recent ESG research, a whopping 84% of organizations are rethinking how they deploy applications as AI enters the picture. It's becoming clear that for AI to deliver real value, it needs quality data behind it. This is pushing IT leaders to take a fresh look at how they approach infrastructure.
Join me as I sit down with Scott Sinclair, Practice Director at Enterprise Strategy Group, to dive into ESG's latest findings on how organizations are reshaping their infrastructure for AI. We unpack some fascinating trends, including:
- Why companies are starting to question if "cloud-first" makes sense for AI
- The real costs of moving data around for AI (spoiler: it's more complex than you think)
- An unexpected plot twist: highly regulated industries are actually leading the AI charge
- Smart ways to start small but build for growth
- Why your data strategy could make or break your AI success
- Getting the right mix of cloud and on-prem for AI workloads
- Real-world tips for building AI infrastructure that won't break the bank
Scott brings hard data and real-world examples to cut through the hype around AI infrastructure. Whether you're just dipping your toes into AI or scaling up existing projects, you'll walk away with practical insights you can actually use.
Subscribe via Spotify: https://open.spotify.com/show/5mKnQYI6PLsnl4RZT2WYdz Subscribe via Apple Podcasts: https://podcasts.apple.com/us/podcast/tech-beyond-the-hype/id1681089332 Subscribe via YouTube Music: https://music.youtube.com/playlist?list=PL5OdmBrO1LpklunLxwIX5AgqfmXQcdC-8&si=Dj5Ut42LxcCiTZPS
Mark as Played
Transcript
Episode Transcript
Available transcripts are automatically generated. Complete accuracy is not guaranteed.
Ana (00:05):
This is Tech Beyond the Hype, the podcast where we talk to experts
and leaders about the latest techand business trends to figure out
what's shaping the future of work.
Welcome back to the show.
I'm Ana, and today we're tacklingone of the biggest challenges facing
businesses in 2024, how to build the rightfoundation for Artificial Intelligence.
(00:26):
Our guest, Scott Sinclair, is PracticeDirector at Enterprise Strategy
Group, and in the interview, he shareseye-opening research on how companies are
navigating the complex decisions betweencloud and on premises AI deployments.
Whether you're just starting your AIjourney or scaling existing solutions,
this episode is packed with practicalinsights that you do not want to miss.
(00:48):
I hope you find it helpful andthat you enjoy the interview
as much as I enjoyed making it.
Now, on with the show!
Scott, thank you so much for joining me.
It is an absolute pleasureto have you all today.
I'm really excited to talk to you aboutAI infrastructure and the way that it's
shaping business decisions and how AIis changing the way that businesses
(01:10):
look at their data infrastructure,but before we dive into that, could you
just introduce yourself a little bitand tell us who you are and what you do?
Scott (01:17):
Yeah, absolutely.
I'm happy to be here.
So as you mentioned, myname is Scott Sinclair.
I'm a practice director withthe Enterprise Strategy Group.
And for those who are listening, whomay not be familiar with the Enterprise
Strategy Group, we're an IT analyst firm.
We do a tremendous amount of research.
As well as strategy work on all facets oftechnology, everything from infrastructure
(01:37):
to application, modernization, to cloud,to security, to Artificial Intelligence,
which we're going to talk about today.
And I'm excited to have this conversation.
I'm going to throw out some ofour research, but, I'm excited
to be here and chat with you.
Ana (01:49):
Excellent.
So before the podcast, I hadn't lookedthrough some of your research that you
just mentioned, and part of it mentionsthe evolution of IT infrastructure.
How would you say that AI isshaping the way that businesses
think about their infrastructure?
Scott (02:04):
It's a great question.
I mean, AI on one hand, it'seasy to think about it as, "oh,
it's just the next big thing."
there's always been these bigthings that, you know, it was cloud.
And then remember when we wereall excited about Bitcoin and,
you know, everything else.
And then cybersecuritycontinues to be a focus.
That being said, it is fascinating whetherwe're talking about machine learning
(02:25):
or now this new advent of generativeAI how quickly nearly every size
organization across nearly every industryis just enamored with the potential
is probably a great way to say that.
And that's just not the, youknow, the, the CIOs or the CTOs,
the CEOs all understand this.
(02:45):
And I think a lot of it was spurred byChat GPT coming out and being really
just the best demonstration of what thepotential of generative AI could be.
And my personal belief is every CEOsaw that and said, "Ooh, I want that.
I want one of those for us."
So what does that mean?
Well, like I mentioned, machine learninghas been around for several years.
(03:06):
Our research into that has showntremendous benefits, tremendous return on
investment for those type of initiatives.
Looking at generative AI in particular,in a recent study, by the end of the
year about 30 percent of organizationsin the study expect to have some
form of generative AI project inproduction, and an additional 33
(03:27):
percent expect to be in pilot or POC.
So organizations are movingvery, very quickly on this.
Now, you ask, how is thisimpacting IT strategy?
How does it impact howwe think about data?
Well, the bottom line is, I mean,it's, it's fueling a number of aspects
because if you think about it, it trulyis, you know, we talk about data, you
(03:51):
know, data being the fuel for modernbusiness or data being the new oil, I
think that came out 15, 20 years ago.
But now it truly is the way in whichwe use data can actually unlock
tremendous new revenue opportunitiesand new competitive differentiation.
And as a result, another researchstat for you, 84 percent of
(04:12):
organizations agreed that thegrowth of AI has us, and including
generative AI, has us reevaluatingour application deployment strategy.
So essentially almostevery organization is a.
We're rethinking the way wedo everything because of AI.
Now, part of that is because at the endof the day, as cloud adoption rose and as
(04:38):
organizations started to go from hybridclouds, multi cloud to now we have lots
of data centers, we have lots of clouds,we've got edge environments, essentially
data's everywhere and placement was oftendriven by what makes sense for the app.
Sometimes it was just drivenby what's available, what makes
sense from a cost standpoint.
Now, if we think about AI andhow we're actually able to train
(04:59):
models on our own data, it getsorganizations thinking about, well,
"which data do we need to train?
Where is it?
Where does it make senseto have the AI live?
Does it make sense to haveit in multiple places?
How do we manage this data?"
And I think that's a lot of what we'reseeing is organizations coming to
grips with the data aspect of it, butin addition to it, it's other things
(05:24):
like access to accelerator technology,like things from NVIDIA, it's access to
"which environments have the best tools,what expertise do I have in house?"
And in addition, the thing thatI've kind of been avoiding,
but saving for last is cost.
We see dramatic differences inthe cost of solutions based on the
location, based on the capabilities,based on the performance.
(05:46):
And this variety on one hand iswonderful for buyers because I have
a lot of choices, but on the otherhand, it's horrible when you're trying
to make a decision because I wantto say nearly, if not every single
it vendor out there, if you come andask, do you have a solution for AI?
Their answer will be yes.
Now, all those answers vary dramatically,but just the fact that there's
(06:09):
so much noise in the marketplace,it is getting very difficult.
Ana (06:11):
Yeah, I can only imagine to summarize what you said, you've
got all this data sitting acrossmultiple different locations.
You know that you have the potential touse that data to unlock revenue streams
or make better business decisions,but having a bunch of different
products on the market available.
As the buyer in that situation, it'skind of like decision paralysis.
How do you know what works based on whatyou've got, and how do you change what
(06:36):
you've got so that it's ready for theproduct that you're hoping to implement?
And I guess to that point, is there anideal infrastructure for AI in 2024?
Is there a data management strategythat's Ideal for integrating AI.
And if so, are there any elements toit, which are surprising or different
to what you might've seen previously?
Scott (06:58):
Oh, yeah.
First off, I want to acknowledge, I lovethe analysis paralysis comment because
that's literally what's happening amonga tremendous number of organizations.
And so actually we see both sides.
So on one hand, we see peoplewith the analysis paralysis.
The other thing we have the, "I don't carehow much it costs, just make it happen."
And those people, as you might expect,whenever we start, when regardless of what
(07:22):
part of our lives we say, I don't carehow much it costs, just make it happen.
Eventually we say, "wait,why does this cost so much?"
And it's like, well, you wantedto move quickly and we just
use the most effective thing.
And now you're questioning us on return.
The few organizations that arepushing back on AI after early
adoption, it's a return standpoint.
It's like, well, what are thereturns been on our investment?
(07:44):
And some of those environments did the,'we don't care how much it costs, just get
started now.' And so there is a balancewe have to find now as it relates to
which environment makes the most sense.
The right answer is unfortunately theanswer no one was here is it depends.
It depends on what you want to do.
It depends on whatyou're trying to achieve.
Now, the big challenge, you know,I mentioned about data, right?
(08:07):
Because data is everywhere.
You know, just some statsbecause, you know, I'm a stats
guy, so I'll throw them out.
80 percent of organizations leveragemore than one public cloud provider.
That's in addition to whateverthey have on premises.
82 percent of organizations agreethat leveraging multiple public cloud
providers delivers strategic benefits.
So we're not in a world whereanyone's ever looking to consolidate
environments anytime soon.
(08:28):
Application data, it's disparate, it'sspread across different environments.
It's going to stay that way.
Now, the challenge, of course,is it's not just the physical
distance between these environmentsand data being spread everywhere.
It's each environment has itsown disparate experience, its own
different tools, requirements,capabilities, everything else.
(08:48):
And often within a company,those are different teams.
You know, we have an AWS team, wehave an Azure team, we have a GCP
team, we have our own internal team.
Now, sometimes there's overlap.
But often the different experiencesmake that even more challenging as
you're trying to develop a solutionthat leverages data everywhere.
Something I didn't mention out ofall of this, and I probably should
(09:09):
have, is I'm talking a lot about data.
When we think about AI, organizationsthat are very early into it may focus
primarily on let's think about training,let's thiknk about inference, let's
build our own AI environment, which, allvery true, you have to do those things.
And those can reside on premises, they canbe in the cloud, they can be at the edge,
they can be at a variety of locations.
The key part, though, to building asuccessful, repeatable, continuous
(09:36):
environment is thinking about thedata pipeline that serves that
training and inferencing environment.
And that's why I'm talking a lot aboutmultiple clouds and and data being
spread everywhere because it's aboutthe importance of building that data
pipeline so that you can continuouslyfeed those investments you've made in
accelerator technology, things like NVIDIAto make sure you're doing the training
(09:58):
and and tuning of the individual models.
Now, the other thing that we've seen is.
So this is what makes it reallydifficult is much of the data
that we want to use in AI hassensitivity or locality requirements.
Things like, "hey, we wantto look at customer data and
see what we can get here.
Maybe we make some sort of customizableresponse as part of a generative
(10:23):
AI based chat bot that can knowessentially, who I am and what my needs
are so I can respond more directly tomy particular issues to do that, it
has to access information about me."
Well, there might be localityrequirements where organizations
want to keep those on premises.
Now, challenge with that is someof the best tools to get started
(10:43):
with AI and do that are locatedon the public cloud providers.
And in fact, when we looked at whyorganizations, how they think about
public cloud providers, ArtificialIntelligence was the most commonly
identified workload that drives someoneto use a different cloud provider
than their primary cloud provider.
That's not to say, for example, thedominant cloud providers are as good
(11:04):
as the secondary cloud providers.
AI is just so important that peoplewill add extra analysis to where
they put that or extra scrutiny.
And it's also one of the top reasons whypeople switch from one cloud provider
to another is because AI and thevarious support for AI is so important.
When it comes to actually deployingAI and where they do it right now,
(11:24):
organizations are deploying ArtificialIntelligence based workloads
everywhere, whether it's edge, publiccloud, co-location on premises.
Now, overall, on average,organizations are more likely to
deploy AI workloads on public cloudsearches than they are on premises.
And that's by a ratio of about two to one.
(11:46):
That being said, when we asked peopleand the majority of organizations, I
said, "look, if you could put your AIenvironment anywhere, where would you be?"
The preference for many organizationsis to keep that on premises.
And so this is, this is where wehave a difference between what people
do versus what they would prefer.
And I think that preference is basedoff of, "we know AI is important.
(12:09):
We know it requires sensitive,important data, and we like the
idea of having better control ofthat also from a cost standpoint."
That being said, there's just arecognition that public cloud tools
are more mature in many cases andcan help people get started faster.
And that time to value is very important.
(12:30):
So as it relates back to anyonelistening, that's getting started in this,
there's multiple factors to consider.
It's thinking about whichdata sets are you planning?
What's the data localityrequirements for those?
And then which tools and experiencedo you have in house to go build?
You know, are youessentially an AWS only shop?
What tools do they have?
(12:51):
Do you have that expertise in house?
Because that's going to playa huge role here as well.
So at the end of the day, it depends.
But often it depends off of whereyour data is, which your own internal
investments have been in the expertiseof your own internal personnel.
Ana (13:08):
Right.
That's really interesting.
So what I'm hearing is you've got thistension between organizations wanting
to bring their data on prem because ofthe control element, but then at the
same time, having a lot more services,a lot more advanced applications
available on the public cloud.
So then there's this kindof toss up between the two.
(13:30):
Trying to balance both and you'vementioned a couple of times cost.
Is it more cost effective to do all thesethings on prem or in the public cloud?
Scott (13:38):
So the fun part is obviously it depends.
That being said, and, and I'mmaking some assumptions, right?
We, we have seen that it can be morecost effective to do it on premises.
Now, part of that assumptionis you have a data center and
data center personnel already.
If you don't, then it's, this is going tobe cheaper in the cloud because you don't
have those existing fixed investments.
(13:59):
The other thing too, that we've seen,and this goes back into how organizations
think about some of the challenges.
So we, we talked about datalocality and, and security and
concerns there and compliance.
All those things lead tocertain data sets, people
wanting those to stay in house.
So movement of data has a cost to it.
Now, whether that's like a dollar'scost or whether that's a cost in risk
(14:22):
or whether it's a cost in time, there'sa, there's a cost of moving data.
So ideally you don't want to do that.
The other thing that on premises providesa couple of things that organizations
that are exploring AI started to figureout is it provides you a little more
control over your cost scalability.
You can make an investment, for example,within AI and keep those costs controlled
(14:50):
rather than basically opening up thecloud to some of your data science teams.
And all of a sudden you find outthat on one hand, you're getting that
increased agility, agility, scalability,which is great from the public cloud.
But on the other hand, you can get thosesurprise bills come in as people are
experimenting with different things.
So what organizations have found outis, "Hey, look, as we get started
(15:11):
with AI, sometimes we may wantto prototype in the cloud, which
can get us to move pretty fast."
Well, if we do that, we need tokeep a tight rein on our cloud
costs and keep that under control.
An alternative to that is if wehave data on premises, we can do
a smaller deployment on premises.
Maybe even sometimes it can be aworkstation with an NVIDIA GPU, just
(15:31):
even to get started on some prototyping.
Do something on premises, that waywe know our investment is fixed and
we can get better control on it, seewhat the outcome is, and then scale
under a more controlled environment.
The other thing that we've seen justin general when it comes to, and I'm
not going to do any sort of vendorcomparison in this, so mileage may vary.
(15:53):
If you're listening to this,always evaluate your own
cost of individual solutions.
That's kind of the fine printin what I'm going to say here.
That being said, when we ask organizationsthat have done the analysis of cost
between on premises and And publiccloud, and for example, when we ask
organizations that are looking to migratea bunch of workloads to the cloud, we
(16:15):
ask them, why don't you, which types ofworkloads do not migrate to the cloud?
Sometimes the rationale is that datalocality, the compliance reasons I
have, that's one of the top ones.
One of the other top ones that weconsistently see is it is more cost
effective to get low latency performanceto large amounts of data on premises.
(16:37):
So we see that oftenshow up in our research.
And that makes sense.
And if you think about it, what is AI?
It is a workload that requires ahigh performance to a lot of data.
So there are a lot of options todeploy this overall from a lower
cost standpoint on premises.
(16:58):
That being said, everything I saidbefore about public cloud having
better tools, better access, betterscale, quality, better time to
value - all that stuff is still true.
So this is part of the balance thatorganizations have to figure out and
manage as they evaluate what they use.
And what that's led to issometimes increased data movement.
(17:19):
75 percent of organizations in ourresearch agree, "hey, look, we've
regularly moved apps or data fromone cloud provider to another."
and that's in addition to allthe movement we see on from on
premises to the public cloud.
And in addition to that, aboutthree quarters of organizations
also say we face challenges withapplication and data portability.
Moving data is difficult.
(17:41):
And so to put it in a nutshell,what organizations have found
out that on premises deliversis better control over costs.
And as you scale what we see in ourresearch, there's a suggestion that on
premises can provide the ability to bemore efficient in their use of cost to
(18:03):
deliver performance for AI workloads.
That being said, there's always a tradeoff because of the tools and capabilities
and the speed at which you can, um,get projects off the ground in AI.
So again, it's a challenge, but Ithink the key takeaway is if we would
have looked at this two years ago andyou said, where should AI be done?
(18:26):
Nearly everyone would say,well, obviously the cloud.
I think now the reaction is,well, actually it depends.
And on premises environmentsactually have several benefits
that should be considered.
Ana (18:39):
Yeah.
I was going to say that actually,because I was thinking as you were
talking - in the pre-AI world,it's the idea of data coming back
on prem was next to unthinkable.
We were all talking cloud, non stop.
Everything was cloud, and italmost seemed as though on premises
stuff, data storage was kind ofgoing out of fashion in a sense.
(19:02):
So it's interesting to see how AI isshifting the way that we think about
infrastructure and where things are kept.
Moving forward a little bit, and I knowthat you're a stats guy, so I might be
putting you outside of your comfort zonewith this question, but if you had to bet
on one emerging technology that's goingto revolutionize AI infrastructure within
the next kind of three to five years.
(19:24):
What would you say that is?
Scott (19:26):
Oh, wow.
Okay.
So to me, this is the big challengeand I didn't provide my background
during the opening, but I spent alot of my time on the infrastructure
side, data storage, which is kindof why we're talking about this.
And as much as I'd love to say, there'sgoing to be some new accelerator
technology that's going to completelychange the world - and there might be,
(19:48):
when I look at the big challenges of AIand I keep coming back to data right now.
Where we see some of the early usecases, the early investments is in
areas where organizations have agood sense on what internal data they
(20:10):
want to use to go train these models.
I should have said this earlier, butthere, there's a number of different
ways that you can build models.
There are many open sourceoff-the-shelf models that you can get.
I think when AI first showed up,the thought was, "well, everyone's
obviously going to train yourown models on your own data."
Now, nobody wants to do that.
That's too expensive.
So what we found is where organizationstend to be leaning towards is you take
(20:34):
an off-the-shelf model and you tune itwith your own data, whether it's through
a retrieval augmented generation or RAGor some of these other tools to tune the
model so basically you can augment it withyour own data so it knows the answers to
the questions you want it to have, butyou're not building a model from scratch.
(20:55):
Now, the reason why I bring thatup is in a world in which every
organization is investing in AI,how do you create differentiation?
Well, how you create differentiationis in your ability to use your data
to augment those off-the-shelf models.
(21:16):
And to me, the big challenge thatwe have as not just people in IT,
but also just as a society, we arenot good at keeping track of data.
We just aren't.
What's funny is, and this is moreanecdotal, but some of the more highly
regulated industries tend to be able toactually make headway into AI faster than
(21:38):
some of the less regulated industries.
Because they've had to putall the scrutiny into data
governance ahead of time.
But if you think about it for manycompanies, we've been treating
data as this giant junk drawer.
The concept of a data lake thatpeople still talk about was big about
five, seven years ago was the idea ofwe'll just throw all your data in one
place and then you can get it later.
Well, the point is, if you can'tunderstand what data you want to train
(22:03):
your model with, and then as that datachanges, understand, okay, this is the
new thing that we need to make surethe model gets trained And you don't
have the right guardrails in placeto do AI, and this gets us more into
the concept of responsible AI, theability to accurately train models.
If a model has a hallucinationor does something that you don't
(22:27):
expect, the ability to quicklydiagnose, why is that, and remediate
it quickly and update it properly.
It's not just gettingthat first model done.
It's about how do you continuouslyimprove this thing over time to
continue to deliver value and theability to do that effectively and
successfully is really what's going toseparate the successful use of AI or
(22:49):
the leaders from just everybody else.
And this is my long way of comingall the way back to your question
of, well, what's going to bethe technology that helps that?
Well, we need better ways to manage andthink about data and data management
tools and data management platforms.
In order to better prep for this worldwhere new data from the business is
(23:15):
going to come in, or we're going toneed to augment it and update our
AI models in order to keep pace.
Infrastructure technology willcontinue to evolve and adapt and
give us more horsepower and, andmore performance and more capacity,
all that stuff's going to happen.
And it will all be important.
But what I think is really necessary hereis something that helps an enterprise
(23:37):
more effectively manage data in theseenvironments to get better at that.
Because at the end of the day, I thinkthat's going to be the big challenge that
organizations face in terms of takingAI from a project and a prototype to
something that is in production that thebusiness can rely on, you know, three,
four, five, six years down the road.
Ana (23:56):
Right.
So some sort of tool that helps withgetting all of the data management
side of things on track so that it'sclear what data is where, what data
is being used for where and how.
Scott (24:09):
That's correct.
And, you know, I'm talkinglike these tools don't exist.
There are tools out therethat do some of these things.
I think we're still very early inwhat these tools can be and the power
and capability that they have as theytranslate out of the data science world
to something that's more accessible.
(24:29):
And I think this is something we'regoing to continue to see more and
more innovation on moving forward.
The way I approach the question isI looked at all the things that we
need to do to make AI or AI enabledworkloads or AI initiatives, part of
a production level enterprise levelbusiness that continues to function.
(24:52):
And the part that spooks me is allthe data management stuff to me.
That's the complex element.
And like I said, there aredefinitely tools out there that
help address it, but it's an areawhere when I talk to companies and
when I talk to organizations, thisis the hard part to figure out.
Ana (25:09):
That makes a lot of sense and resonates a lot.
On a personal level, I've workedin a number of companies in the
past kind of 10 years, and datais always something that is messy.
So it makes a lot of sense that this issomething that businesses just generally,
up until now, there hasn't really beena reason - unless you're, like you said,
one of those sectors where data really isheavily regulated, and there's a lot of
rules around how and what you store andwhere you're not in that kind of sector.
(25:34):
It makes a lot of sensethat this is the big issue.
And how do we get our data ready sothat we can use these tools and make
sure that they're not hallucinating andcoming up with things that make no sense,
because if you're working with a datalake, like you said, how do you check
that the data that the AI model thatyou've implemented is actually correct.
So the answer isn't just rubbish.
(25:55):
It's impossible.
Scott (25:55):
Exactly.
And it's funny, I was having aconversation about this where, uh, you
know, differing opinions and someone inthe AI space discreet and said, yo, we
have lots of tools that today people useand there are processes to go manage this.
One of the things that example was,you know, you'll look at CRM tools
or things like, like Salesforce, forexample, where you're able to input a
whole bunch of data, we have processeswhere, you know, within databases.
(26:17):
And where we can get this analysis outand my response was, so even the stuff
that you're doing right now in Salesforce,you can trust that all the time.
Oh, no, no.
Sometimes people put in the wrongthings and then we get into.
Oh, no, we're not trackingmeetings properly.
And sometimes there's a deal inthere that doesn't really make sense.
And I'm like, look, all of that stuff thatwe just kind of wave our hands with and
(26:38):
kind of figure out as humans that we seein the tools we use today, that we sit
down, ah, that one, I don't really trust.
Let me go in and figure it out.
Oh, someone didn't enter it correctly.
All that stuff is, you know,the, I'm, I'm kind of mixing, you
know, apples and oranges here.
But it's those types of things thatcreate the risk for an AI project to go,
(26:59):
Oh, no, I'm, I'm being trained on that.
Cause I'm treating them all equally.
Right.
So even though as an organization, manycompanies may be, Oh, we've solved that.
I would argue, well, maybe you've solvedit 80 percent of the way, or maybe you
sold it 90 percent of the way, but.
It's that extra little bit that'sgoing to actually, uh, go a long way
to determining the actual experiencethat the AI solution provides.
Ana (27:21):
Sure, that makes a lot of sense.
And thinking about these masses ofdata, and we were talking about data
mobility, we touched on it earlier.
How are the cutting edge organizationslooking at data mobility?
What are they doing to solve thechallenge of moving massive amounts
of data in a way that's AI efficient?
Scott (27:40):
Yeah, so there's a couple things on that.
I would say A lot has been messagedaround the idea of don't move the
data, put the AI with the data.
And that's what's led to a lot ofthis desire or resurgence of on
premises data center type things.
Okay, let's say instead of moving allof our data to the public cloud, for
(28:00):
example, let's put it where the data is.
That being said, if you have data inthe cloud, let's put the AI there.
That is very nice in theory, and I thinkit's a good practice to get started.
But one of the things that has continuedto show up over and over again in
all the research we do, whether it'son AI or anything else, is just how
spread out data environments are.
(28:20):
So what I recommend is in addition to yes,try not to move data because every time
you move data, there's cost risk time,all sorts of things associated with that.
Also, it's important to investin whether data management tools
as well as data storage tools.
Can span locations and provideconsistency of experience across
(28:44):
location and consistency offunctionality across location.
What we need to get out of is thisworld of where if data lives on AWS,
I have to deal with a different teamthan if it lives on Microsoft Azure.
Cause that's going to kill us over time.
You can't scale that way.
You can't hire that way.
(29:05):
It's very, very difficultto actually deal with that.
You need tools that are consistent enough.
And easy enough and automated enoughto where you can have one team get
a more holistic view over not justyour data, but also the storage in
how you're securing and protectingthat data across these environments.
(29:26):
And I think that'sgoing to be a big thing.
I talked a lot about data managementand how important that is.
I think.
Something else that is going to bekey is tools that are able to help
facilitate the movement of data.
There's some storage tools out there,for example, that can auto migrate
data or provide, for example, youknow, for thinking about unstructured
data or files, a global namespace sothat if I'm running an AI project in
(29:51):
the cloud, I can actually see datathat resides in multiple locations.
And manage that effectively from thatstandpoint, at the end of the day, I
think so much of it is going to come fromjust thinking about the data pipeline
from a multi location standpoint.
And simplifying that, that being said, allthis stuff I talked about is about getting
(30:16):
your data environment to where you're ina strong place for long term AI success.
I think as it comes to organizationsthat are getting started, I think
what really matters there, and this isgoing to sound like I'm contradicting
myself, is start very small.
Start very small with a dataset that you understand.
(30:37):
And pick a use case that is very muchinternal so you can get it, you know, in
business, we call these quick wins, right?
You want a quick win.
You want something where you delivervalue quickly to your executive
team where you can measure it andyou have a cost under control.
And that's what led to, youknow, I mentioned before, we do
see organizations that are doingsome prototyping on premises.
(31:01):
And that's what's leading to that,you know, let's get a workstation.
Let's get a, you know, a video GPU.
Let's get some data out here andlet's see if we can get something
prototyped and up and running quickly.
And I think that's essential becauseall this other stuff I talked about,
which is about getting your data stateand into where you're doing proper
hygiene and secure, you're able tomove data back and forth, that's hard.
(31:22):
It is very difficult to do.
And there's tools out there to helpyou to, you know, use an analogy here.
That's kind of like boiling the ocean.
What you want to do isdon't start with that.
Start with getting a small project up andrunning and then think through, okay, the
data for that, how do we manage that dataenvironment and then slowly, gradually
(31:45):
expand that to encompass the other in dataenvironments as your AI initiative scale.
And I think that's something else thatI wanted to make sure I brought up
because I don't want people leavinghere saying, Oh, before we start AI,
I got to get my entire data estate inunder control, which honestly, it's not.
You do, but that's goingto take a long time.
Start small and grow.
Ana (32:03):
It's almost like all of the stuff that we've been talking
about when it comes to datamanagement and data governance.
If you're someone who's out therestarting off from day one, that's
kind of running before you can walk.
You're thinking about those things.
You need to start off with.
The basics.
Get to know, understand how the, howto integrate AI in a small dentist
set so that you're then able toprogress into more complex challenges
(32:26):
as you are learning and developing
Scott (32:28):
that.
That's a great way to put it.
Uh, and actually, I think you broughtup a really interesting point, which is
something that I think we all naturallyassume, but is so important we need to
state it, is as you are learning withAI and you start those initial projects.
Make sure that you capture what those bestpractices are for managing data so that
(32:51):
you can carry it on to the next projectand continue to expand that because it's
not just, Hey, these are the right tools.
This is how you do it, but theseare the best practices to create
that good hygiene, so to speak,whether it comes into how you
manage your data as you scale.
Ana (33:06):
Awesome.
That's all the questionsI had to ask you today.
So thank you so much for joining me.
It's been an absolutepleasure having you on.
I had a really great time.
Thank you.
Speak to you soon.
So that was Scott Sinclairsharing invaluable insights on
the future of AI infrastructure.
One thing is clear, as more businesseslook to implement AI into their workloads,
the way we think about data and datamanagement is changing faster than ever.
(33:29):
And staying ahead of these changesis going to be critical in the
future for businesses to succeed.
To keep up with the latest business andtechnology trends, make sure to like,
rate, and subscribe to Tech Beyond theHype, wherever you listen to podcasts.
Tech Beyond the Hype is a TechTarget original production.
This is Tech Beyond the Hype, the podcast where we talk to experts
and leaders about the latest techand business trends to figure out
what's shaping the future of work.
Welcome back to the show.
I'm Ana, and today we're tacklingone of the biggest challenges facing
businesses in 2024, how to build the rightfoundation for Artificial Intelligence.
(00:26):
Our guest, Scott Sinclair, is PracticeDirector at Enterprise Strategy
Group, and in the interview, he shareseye-opening research on how companies are
navigating the complex decisions betweencloud and on premises AI deployments.
Whether you're just starting your AIjourney or scaling existing solutions,
this episode is packed with practicalinsights that you do not want to miss.
(00:48):
I hope you find it helpful andthat you enjoy the interview
as much as I enjoyed making it.
Now, on with the show!
Scott, thank you so much for joining me.
It is an absolute pleasureto have you all today.
I'm really excited to talk to you aboutAI infrastructure and the way that it's
shaping business decisions and how AIis changing the way that businesses
(01:10):
look at their data infrastructure,but before we dive into that, could you
just introduce yourself a little bitand tell us who you are and what you do?
Scott (01:17):
Yeah, absolutely.
I'm happy to be here.
So as you mentioned, myname is Scott Sinclair.
I'm a practice director withthe Enterprise Strategy Group.
And for those who are listening, whomay not be familiar with the Enterprise
Strategy Group, we're an IT analyst firm.
We do a tremendous amount of research.
As well as strategy work on all facets oftechnology, everything from infrastructure
(01:37):
to application, modernization, to cloud,to security, to Artificial Intelligence,
which we're going to talk about today.
And I'm excited to have this conversation.
I'm going to throw out some ofour research, but, I'm excited
to be here and chat with you.
Ana (01:49):
Excellent.
So before the podcast, I hadn't lookedthrough some of your research that you
just mentioned, and part of it mentionsthe evolution of IT infrastructure.
How would you say that AI isshaping the way that businesses
think about their infrastructure?
Scott (02:04):
It's a great question.
I mean, AI on one hand, it'seasy to think about it as, "oh,
it's just the next big thing."
there's always been these bigthings that, you know, it was cloud.
And then remember when we wereall excited about Bitcoin and,
you know, everything else.
And then cybersecuritycontinues to be a focus.
That being said, it is fascinating whetherwe're talking about machine learning
(02:25):
or now this new advent of generativeAI how quickly nearly every size
organization across nearly every industryis just enamored with the potential
is probably a great way to say that.
And that's just not the, youknow, the, the CIOs or the CTOs,
the CEOs all understand this.
(02:45):
And I think a lot of it was spurred byChat GPT coming out and being really
just the best demonstration of what thepotential of generative AI could be.
And my personal belief is every CEOsaw that and said, "Ooh, I want that.
I want one of those for us."
So what does that mean?
Well, like I mentioned, machine learninghas been around for several years.
(03:06):
Our research into that has showntremendous benefits, tremendous return on
investment for those type of initiatives.
Looking at generative AI in particular,in a recent study, by the end of the
year about 30 percent of organizationsin the study expect to have some
form of generative AI project inproduction, and an additional 33
(03:27):
percent expect to be in pilot or POC.
So organizations are movingvery, very quickly on this.
Now, you ask, how is thisimpacting IT strategy?
How does it impact howwe think about data?
Well, the bottom line is, I mean,it's, it's fueling a number of aspects
because if you think about it, it trulyis, you know, we talk about data, you
(03:51):
know, data being the fuel for modernbusiness or data being the new oil, I
think that came out 15, 20 years ago.
But now it truly is the way in whichwe use data can actually unlock
tremendous new revenue opportunitiesand new competitive differentiation.
And as a result, another researchstat for you, 84 percent of
(04:12):
organizations agreed that thegrowth of AI has us, and including
generative AI, has us reevaluatingour application deployment strategy.
So essentially almostevery organization is a.
We're rethinking the way wedo everything because of AI.
Now, part of that is because at the endof the day, as cloud adoption rose and as
(04:38):
organizations started to go from hybridclouds, multi cloud to now we have lots
of data centers, we have lots of clouds,we've got edge environments, essentially
data's everywhere and placement was oftendriven by what makes sense for the app.
Sometimes it was just drivenby what's available, what makes
sense from a cost standpoint.
Now, if we think about AI andhow we're actually able to train
(04:59):
models on our own data, it getsorganizations thinking about, well,
"which data do we need to train?
Where is it?
Where does it make senseto have the AI live?
Does it make sense to haveit in multiple places?
How do we manage this data?"
And I think that's a lot of what we'reseeing is organizations coming to
grips with the data aspect of it, butin addition to it, it's other things
(05:24):
like access to accelerator technology,like things from NVIDIA, it's access to
"which environments have the best tools,what expertise do I have in house?"
And in addition, the thing thatI've kind of been avoiding,
but saving for last is cost.
We see dramatic differences inthe cost of solutions based on the
location, based on the capabilities,based on the performance.
(05:46):
And this variety on one hand iswonderful for buyers because I have
a lot of choices, but on the otherhand, it's horrible when you're trying
to make a decision because I wantto say nearly, if not every single
it vendor out there, if you come andask, do you have a solution for AI?
Their answer will be yes.
Now, all those answers vary dramatically,but just the fact that there's
(06:09):
so much noise in the marketplace,it is getting very difficult.
Ana (06:11):
Yeah, I can only imagine to summarize what you said, you've
got all this data sitting acrossmultiple different locations.
You know that you have the potential touse that data to unlock revenue streams
or make better business decisions,but having a bunch of different
products on the market available.
As the buyer in that situation, it'skind of like decision paralysis.
How do you know what works based on whatyou've got, and how do you change what
(06:36):
you've got so that it's ready for theproduct that you're hoping to implement?
And I guess to that point, is there anideal infrastructure for AI in 2024?
Is there a data management strategythat's Ideal for integrating AI.
And if so, are there any elements toit, which are surprising or different
to what you might've seen previously?
Scott (06:58):
Oh, yeah.
First off, I want to acknowledge, I lovethe analysis paralysis comment because
that's literally what's happening amonga tremendous number of organizations.
And so actually we see both sides.
So on one hand, we see peoplewith the analysis paralysis.
The other thing we have the, "I don't carehow much it costs, just make it happen."
And those people, as you might expect,whenever we start, when regardless of what
(07:22):
part of our lives we say, I don't carehow much it costs, just make it happen.
Eventually we say, "wait,why does this cost so much?"
And it's like, well, you wantedto move quickly and we just
use the most effective thing.
And now you're questioning us on return.
The few organizations that arepushing back on AI after early
adoption, it's a return standpoint.
It's like, well, what are thereturns been on our investment?
(07:44):
And some of those environments did the,'we don't care how much it costs, just get
started now.' And so there is a balancewe have to find now as it relates to
which environment makes the most sense.
The right answer is unfortunately theanswer no one was here is it depends.
It depends on what you want to do.
It depends on whatyou're trying to achieve.
Now, the big challenge, you know,I mentioned about data, right?
(08:07):
Because data is everywhere.
You know, just some statsbecause, you know, I'm a stats
guy, so I'll throw them out.
80 percent of organizations leveragemore than one public cloud provider.
That's in addition to whateverthey have on premises.
82 percent of organizations agreethat leveraging multiple public cloud
providers delivers strategic benefits.
So we're not in a world whereanyone's ever looking to consolidate
environments anytime soon.
(08:28):
Application data, it's disparate, it'sspread across different environments.
It's going to stay that way.
Now, the challenge, of course,is it's not just the physical
distance between these environmentsand data being spread everywhere.
It's each environment has itsown disparate experience, its own
different tools, requirements,capabilities, everything else.
(08:48):
And often within a company,those are different teams.
You know, we have an AWS team, wehave an Azure team, we have a GCP
team, we have our own internal team.
Now, sometimes there's overlap.
But often the different experiencesmake that even more challenging as
you're trying to develop a solutionthat leverages data everywhere.
Something I didn't mention out ofall of this, and I probably should
(09:09):
have, is I'm talking a lot about data.
When we think about AI, organizationsthat are very early into it may focus
primarily on let's think about training,let's thiknk about inference, let's
build our own AI environment, which, allvery true, you have to do those things.
And those can reside on premises, they canbe in the cloud, they can be at the edge,
they can be at a variety of locations.
The key part, though, to building asuccessful, repeatable, continuous
(09:36):
environment is thinking about thedata pipeline that serves that
training and inferencing environment.
And that's why I'm talking a lot aboutmultiple clouds and and data being
spread everywhere because it's aboutthe importance of building that data
pipeline so that you can continuouslyfeed those investments you've made in
accelerator technology, things like NVIDIAto make sure you're doing the training
(09:58):
and and tuning of the individual models.
Now, the other thing that we've seen is.
So this is what makes it reallydifficult is much of the data
that we want to use in AI hassensitivity or locality requirements.
Things like, "hey, we wantto look at customer data and
see what we can get here.
Maybe we make some sort of customizableresponse as part of a generative
(10:23):
AI based chat bot that can knowessentially, who I am and what my needs
are so I can respond more directly tomy particular issues to do that, it
has to access information about me."
Well, there might be localityrequirements where organizations
want to keep those on premises.
Now, challenge with that is someof the best tools to get started
(10:43):
with AI and do that are locatedon the public cloud providers.
And in fact, when we looked at whyorganizations, how they think about
public cloud providers, ArtificialIntelligence was the most commonly
identified workload that drives someoneto use a different cloud provider
than their primary cloud provider.
That's not to say, for example, thedominant cloud providers are as good
(11:04):
as the secondary cloud providers.
AI is just so important that peoplewill add extra analysis to where
they put that or extra scrutiny.
And it's also one of the top reasons whypeople switch from one cloud provider
to another is because AI and thevarious support for AI is so important.
When it comes to actually deployingAI and where they do it right now,
(11:24):
organizations are deploying ArtificialIntelligence based workloads
everywhere, whether it's edge, publiccloud, co-location on premises.
Now, overall, on average,organizations are more likely to
deploy AI workloads on public cloudsearches than they are on premises.
And that's by a ratio of about two to one.
(11:46):
That being said, when we asked peopleand the majority of organizations, I
said, "look, if you could put your AIenvironment anywhere, where would you be?"
The preference for many organizationsis to keep that on premises.
And so this is, this is where wehave a difference between what people
do versus what they would prefer.
And I think that preference is basedoff of, "we know AI is important.
(12:09):
We know it requires sensitive,important data, and we like the
idea of having better control ofthat also from a cost standpoint."
That being said, there's just arecognition that public cloud tools
are more mature in many cases andcan help people get started faster.
And that time to value is very important.
(12:30):
So as it relates back to anyonelistening, that's getting started in this,
there's multiple factors to consider.
It's thinking about whichdata sets are you planning?
What's the data localityrequirements for those?
And then which tools and experiencedo you have in house to go build?
You know, are youessentially an AWS only shop?
What tools do they have?
(12:51):
Do you have that expertise in house?
Because that's going to playa huge role here as well.
So at the end of the day, it depends.
But often it depends off of whereyour data is, which your own internal
investments have been in the expertiseof your own internal personnel.
Ana (13:08):
Right.
That's really interesting.
So what I'm hearing is you've got thistension between organizations wanting
to bring their data on prem because ofthe control element, but then at the
same time, having a lot more services,a lot more advanced applications
available on the public cloud.
So then there's this kindof toss up between the two.
(13:30):
Trying to balance both and you'vementioned a couple of times cost.
Is it more cost effective to do all thesethings on prem or in the public cloud?
Scott (13:38):
So the fun part is obviously it depends.
That being said, and, and I'mmaking some assumptions, right?
We, we have seen that it can be morecost effective to do it on premises.
Now, part of that assumptionis you have a data center and
data center personnel already.
If you don't, then it's, this is going tobe cheaper in the cloud because you don't
have those existing fixed investments.
(13:59):
The other thing too, that we've seen,and this goes back into how organizations
think about some of the challenges.
So we, we talked about datalocality and, and security and
concerns there and compliance.
All those things lead tocertain data sets, people
wanting those to stay in house.
So movement of data has a cost to it.
Now, whether that's like a dollar'scost or whether that's a cost in risk
(14:22):
or whether it's a cost in time, there'sa, there's a cost of moving data.
So ideally you don't want to do that.
The other thing that on premises providesa couple of things that organizations
that are exploring AI started to figureout is it provides you a little more
control over your cost scalability.
You can make an investment, for example,within AI and keep those costs controlled
(14:50):
rather than basically opening up thecloud to some of your data science teams.
And all of a sudden you find outthat on one hand, you're getting that
increased agility, agility, scalability,which is great from the public cloud.
But on the other hand, you can get thosesurprise bills come in as people are
experimenting with different things.
So what organizations have found outis, "Hey, look, as we get started
(15:11):
with AI, sometimes we may wantto prototype in the cloud, which
can get us to move pretty fast."
Well, if we do that, we need tokeep a tight rein on our cloud
costs and keep that under control.
An alternative to that is if wehave data on premises, we can do
a smaller deployment on premises.
Maybe even sometimes it can be aworkstation with an NVIDIA GPU, just
(15:31):
even to get started on some prototyping.
Do something on premises, that waywe know our investment is fixed and
we can get better control on it, seewhat the outcome is, and then scale
under a more controlled environment.
The other thing that we've seen justin general when it comes to, and I'm
not going to do any sort of vendorcomparison in this, so mileage may vary.
(15:53):
If you're listening to this,always evaluate your own
cost of individual solutions.
That's kind of the fine printin what I'm going to say here.
That being said, when we ask organizationsthat have done the analysis of cost
between on premises and And publiccloud, and for example, when we ask
organizations that are looking to migratea bunch of workloads to the cloud, we
(16:15):
ask them, why don't you, which types ofworkloads do not migrate to the cloud?
Sometimes the rationale is that datalocality, the compliance reasons I
have, that's one of the top ones.
One of the other top ones that weconsistently see is it is more cost
effective to get low latency performanceto large amounts of data on premises.
(16:37):
So we see that oftenshow up in our research.
And that makes sense.
And if you think about it, what is AI?
It is a workload that requires ahigh performance to a lot of data.
So there are a lot of options todeploy this overall from a lower
cost standpoint on premises.
(16:58):
That being said, everything I saidbefore about public cloud having
better tools, better access, betterscale, quality, better time to
value - all that stuff is still true.
So this is part of the balance thatorganizations have to figure out and
manage as they evaluate what they use.
And what that's led to issometimes increased data movement.
(17:19):
75 percent of organizations in ourresearch agree, "hey, look, we've
regularly moved apps or data fromone cloud provider to another."
and that's in addition to allthe movement we see on from on
premises to the public cloud.
And in addition to that, aboutthree quarters of organizations
also say we face challenges withapplication and data portability.
Moving data is difficult.
(17:41):
And so to put it in a nutshell,what organizations have found
out that on premises deliversis better control over costs.
And as you scale what we see in ourresearch, there's a suggestion that on
premises can provide the ability to bemore efficient in their use of cost to
(18:03):
deliver performance for AI workloads.
That being said, there's always a tradeoff because of the tools and capabilities
and the speed at which you can, um,get projects off the ground in AI.
So again, it's a challenge, but Ithink the key takeaway is if we would
have looked at this two years ago andyou said, where should AI be done?
(18:26):
Nearly everyone would say,well, obviously the cloud.
I think now the reaction is,well, actually it depends.
And on premises environmentsactually have several benefits
that should be considered.
Ana (18:39):
Yeah.
I was going to say that actually,because I was thinking as you were
talking - in the pre-AI world,it's the idea of data coming back
on prem was next to unthinkable.
We were all talking cloud, non stop.
Everything was cloud, and italmost seemed as though on premises
stuff, data storage was kind ofgoing out of fashion in a sense.
(19:02):
So it's interesting to see how AI isshifting the way that we think about
infrastructure and where things are kept.
Moving forward a little bit, and I knowthat you're a stats guy, so I might be
putting you outside of your comfort zonewith this question, but if you had to bet
on one emerging technology that's goingto revolutionize AI infrastructure within
the next kind of three to five years.
(19:24):
What would you say that is?
Scott (19:26):
Oh, wow.
Okay.
So to me, this is the big challengeand I didn't provide my background
during the opening, but I spent alot of my time on the infrastructure
side, data storage, which is kindof why we're talking about this.
And as much as I'd love to say, there'sgoing to be some new accelerator
technology that's going to completelychange the world - and there might be,
(19:48):
when I look at the big challenges of AIand I keep coming back to data right now.
Where we see some of the early usecases, the early investments is in
areas where organizations have agood sense on what internal data they
(20:10):
want to use to go train these models.
I should have said this earlier, butthere, there's a number of different
ways that you can build models.
There are many open sourceoff-the-shelf models that you can get.
I think when AI first showed up,the thought was, "well, everyone's
obviously going to train yourown models on your own data."
Now, nobody wants to do that.
That's too expensive.
So what we found is where organizationstend to be leaning towards is you take
(20:34):
an off-the-shelf model and you tune itwith your own data, whether it's through
a retrieval augmented generation or RAGor some of these other tools to tune the
model so basically you can augment it withyour own data so it knows the answers to
the questions you want it to have, butyou're not building a model from scratch.
(20:55):
Now, the reason why I bring thatup is in a world in which every
organization is investing in AI,how do you create differentiation?
Well, how you create differentiationis in your ability to use your data
to augment those off-the-shelf models.
(21:16):
And to me, the big challenge thatwe have as not just people in IT,
but also just as a society, we arenot good at keeping track of data.
We just aren't.
What's funny is, and this is moreanecdotal, but some of the more highly
regulated industries tend to be able toactually make headway into AI faster than
(21:38):
some of the less regulated industries.
Because they've had to putall the scrutiny into data
governance ahead of time.
But if you think about it for manycompanies, we've been treating
data as this giant junk drawer.
The concept of a data lake thatpeople still talk about was big about
five, seven years ago was the idea ofwe'll just throw all your data in one
place and then you can get it later.
Well, the point is, if you can'tunderstand what data you want to train
(22:03):
your model with, and then as that datachanges, understand, okay, this is the
new thing that we need to make surethe model gets trained And you don't
have the right guardrails in placeto do AI, and this gets us more into
the concept of responsible AI, theability to accurately train models.
If a model has a hallucinationor does something that you don't
(22:27):
expect, the ability to quicklydiagnose, why is that, and remediate
it quickly and update it properly.
It's not just gettingthat first model done.
It's about how do you continuouslyimprove this thing over time to
continue to deliver value and theability to do that effectively and
successfully is really what's going toseparate the successful use of AI or
(22:49):
the leaders from just everybody else.
And this is my long way of comingall the way back to your question
of, well, what's going to bethe technology that helps that?
Well, we need better ways to manage andthink about data and data management
tools and data management platforms.
In order to better prep for this worldwhere new data from the business is
(23:15):
going to come in, or we're going toneed to augment it and update our
AI models in order to keep pace.
Infrastructure technology willcontinue to evolve and adapt and
give us more horsepower and, andmore performance and more capacity,
all that stuff's going to happen.
And it will all be important.
But what I think is really necessary hereis something that helps an enterprise
(23:37):
more effectively manage data in theseenvironments to get better at that.
Because at the end of the day, I thinkthat's going to be the big challenge that
organizations face in terms of takingAI from a project and a prototype to
something that is in production that thebusiness can rely on, you know, three,
four, five, six years down the road.
Ana (23:56):
Right.
So some sort of tool that helps withgetting all of the data management
side of things on track so that it'sclear what data is where, what data
is being used for where and how.
Scott (24:09):
That's correct.
And, you know, I'm talkinglike these tools don't exist.
There are tools out therethat do some of these things.
I think we're still very early inwhat these tools can be and the power
and capability that they have as theytranslate out of the data science world
to something that's more accessible.
(24:29):
And I think this is something we'regoing to continue to see more and
more innovation on moving forward.
The way I approach the question isI looked at all the things that we
need to do to make AI or AI enabledworkloads or AI initiatives, part of
a production level enterprise levelbusiness that continues to function.
(24:52):
And the part that spooks me is allthe data management stuff to me.
That's the complex element.
And like I said, there aredefinitely tools out there that
help address it, but it's an areawhere when I talk to companies and
when I talk to organizations, thisis the hard part to figure out.
Ana (25:09):
That makes a lot of sense and resonates a lot.
On a personal level, I've workedin a number of companies in the
past kind of 10 years, and datais always something that is messy.
So it makes a lot of sense that this issomething that businesses just generally,
up until now, there hasn't really beena reason - unless you're, like you said,
one of those sectors where data really isheavily regulated, and there's a lot of
rules around how and what you store andwhere you're not in that kind of sector.
(25:34):
It makes a lot of sensethat this is the big issue.
And how do we get our data ready sothat we can use these tools and make
sure that they're not hallucinating andcoming up with things that make no sense,
because if you're working with a datalake, like you said, how do you check
that the data that the AI model thatyou've implemented is actually correct.
So the answer isn't just rubbish.
(25:55):
It's impossible.
Scott (25:55):
Exactly.
And it's funny, I was having aconversation about this where, uh, you
know, differing opinions and someone inthe AI space discreet and said, yo, we
have lots of tools that today people useand there are processes to go manage this.
One of the things that example was,you know, you'll look at CRM tools
or things like, like Salesforce, forexample, where you're able to input a
whole bunch of data, we have processeswhere, you know, within databases.
(26:17):
And where we can get this analysis outand my response was, so even the stuff
that you're doing right now in Salesforce,you can trust that all the time.
Oh, no, no.
Sometimes people put in the wrongthings and then we get into.
Oh, no, we're not trackingmeetings properly.
And sometimes there's a deal inthere that doesn't really make sense.
And I'm like, look, all of that stuff thatwe just kind of wave our hands with and
(26:38):
kind of figure out as humans that we seein the tools we use today, that we sit
down, ah, that one, I don't really trust.
Let me go in and figure it out.
Oh, someone didn't enter it correctly.
All that stuff is, you know,the, I'm, I'm kind of mixing, you
know, apples and oranges here.
But it's those types of things thatcreate the risk for an AI project to go,
(26:59):
Oh, no, I'm, I'm being trained on that.
Cause I'm treating them all equally.
Right.
So even though as an organization, manycompanies may be, Oh, we've solved that.
I would argue, well, maybe you've solvedit 80 percent of the way, or maybe you
sold it 90 percent of the way, but.
It's that extra little bit that'sgoing to actually, uh, go a long way
to determining the actual experiencethat the AI solution provides.
Ana (27:21):
Sure, that makes a lot of sense.
And thinking about these masses ofdata, and we were talking about data
mobility, we touched on it earlier.
How are the cutting edge organizationslooking at data mobility?
What are they doing to solve thechallenge of moving massive amounts
of data in a way that's AI efficient?
Scott (27:40):
Yeah, so there's a couple things on that.
I would say A lot has been messagedaround the idea of don't move the
data, put the AI with the data.
And that's what's led to a lot ofthis desire or resurgence of on
premises data center type things.
Okay, let's say instead of moving allof our data to the public cloud, for
(28:00):
example, let's put it where the data is.
That being said, if you have data inthe cloud, let's put the AI there.
That is very nice in theory, and I thinkit's a good practice to get started.
But one of the things that has continuedto show up over and over again in
all the research we do, whether it'son AI or anything else, is just how
spread out data environments are.
(28:20):
So what I recommend is in addition to yes,try not to move data because every time
you move data, there's cost risk time,all sorts of things associated with that.
Also, it's important to investin whether data management tools
as well as data storage tools.
Can span locations and provideconsistency of experience across
(28:44):
location and consistency offunctionality across location.
What we need to get out of is thisworld of where if data lives on AWS,
I have to deal with a different teamthan if it lives on Microsoft Azure.
Cause that's going to kill us over time.
You can't scale that way.
You can't hire that way.
(29:05):
It's very, very difficultto actually deal with that.
You need tools that are consistent enough.
And easy enough and automated enoughto where you can have one team get
a more holistic view over not justyour data, but also the storage in
how you're securing and protectingthat data across these environments.
(29:26):
And I think that'sgoing to be a big thing.
I talked a lot about data managementand how important that is.
I think.
Something else that is going to bekey is tools that are able to help
facilitate the movement of data.
There's some storage tools out there,for example, that can auto migrate
data or provide, for example, youknow, for thinking about unstructured
data or files, a global namespace sothat if I'm running an AI project in
(29:51):
the cloud, I can actually see datathat resides in multiple locations.
And manage that effectively from thatstandpoint, at the end of the day, I
think so much of it is going to come fromjust thinking about the data pipeline
from a multi location standpoint.
And simplifying that, that being said, allthis stuff I talked about is about getting
(30:16):
your data environment to where you're ina strong place for long term AI success.
I think as it comes to organizationsthat are getting started, I think
what really matters there, and this isgoing to sound like I'm contradicting
myself, is start very small.
Start very small with a dataset that you understand.
(30:37):
And pick a use case that is very muchinternal so you can get it, you know, in
business, we call these quick wins, right?
You want a quick win.
You want something where you delivervalue quickly to your executive
team where you can measure it andyou have a cost under control.
And that's what led to, youknow, I mentioned before, we do
see organizations that are doingsome prototyping on premises.
(31:01):
And that's what's leading to that,you know, let's get a workstation.
Let's get a, you know, a video GPU.
Let's get some data out here andlet's see if we can get something
prototyped and up and running quickly.
And I think that's essential becauseall this other stuff I talked about,
which is about getting your data stateand into where you're doing proper
hygiene and secure, you're able tomove data back and forth, that's hard.
(31:22):
It is very difficult to do.
And there's tools out there to helpyou to, you know, use an analogy here.
That's kind of like boiling the ocean.
What you want to do isdon't start with that.
Start with getting a small project up andrunning and then think through, okay, the
data for that, how do we manage that dataenvironment and then slowly, gradually
(31:45):
expand that to encompass the other in dataenvironments as your AI initiative scale.
And I think that's something else thatI wanted to make sure I brought up
because I don't want people leavinghere saying, Oh, before we start AI,
I got to get my entire data estate inunder control, which honestly, it's not.
You do, but that's goingto take a long time.
Start small and grow.
Ana (32:03):
It's almost like all of the stuff that we've been talking
about when it comes to datamanagement and data governance.
If you're someone who's out therestarting off from day one, that's
kind of running before you can walk.
You're thinking about those things.
You need to start off with.
The basics.
Get to know, understand how the, howto integrate AI in a small dentist
set so that you're then able toprogress into more complex challenges
(32:26):
as you are learning and developing
Scott (32:28):
that.
That's a great way to put it.
Uh, and actually, I think you broughtup a really interesting point, which is
something that I think we all naturallyassume, but is so important we need to
state it, is as you are learning withAI and you start those initial projects.
Make sure that you capture what those bestpractices are for managing data so that
(32:51):
you can carry it on to the next projectand continue to expand that because it's
not just, Hey, these are the right tools.
This is how you do it, but theseare the best practices to create
that good hygiene, so to speak,whether it comes into how you
manage your data as you scale.
Ana (33:06):
Awesome.
That's all the questionsI had to ask you today.
So thank you so much for joining me.
It's been an absolutepleasure having you on.
I had a really great time.
Thank you.
Speak to you soon.
So that was Scott Sinclairsharing invaluable insights on
the future of AI infrastructure.
One thing is clear, as more businesseslook to implement AI into their workloads,
the way we think about data and datamanagement is changing faster than ever.
(33:29):
And staying ahead of these changesis going to be critical in the
future for businesses to succeed.
To keep up with the latest business andtechnology trends, make sure to like,
rate, and subscribe to Tech Beyond theHype, wherever you listen to podcasts.
Tech Beyond the Hype is a TechTarget original production.
Popular Podcasts
United States of Kennedy
United States of Kennedy is a podcast about our cultural fascination with the Kennedy dynasty. Every week, hosts Lyra Smith and George Civeris go into one aspect of the Kennedy story.
Dateline NBC
Current and classic episodes, featuring compelling true-crime mysteries, powerful documentaries and in-depth investigations. Follow now to get the latest episodes of Dateline NBC completely free, or subscribe to Dateline Premium for ad-free listening and exclusive bonus content: DatelinePremium.com
Bookmarked by Reese's Book Club
Welcome to Bookmarked by Reese’s Book Club — the podcast where great stories, bold women, and irresistible conversations collide! Hosted by award-winning journalist Danielle Robay, each week new episodes balance thoughtful literary insight with the fervor of buzzy book trends, pop culture and more. Bookmarked brings together celebrities, tastemakers, influencers and authors from Reese's Book Club and beyond to share stories that transcend the page. Pull up a chair. You’re not just listening — you’re part of the conversation.