July 30, 2026 • 18 mins
In this episode of "Full Tech Ahead," host Amanda Razani interviews DJ Spry, Head of Product for Aria Networks. The discussion focuses on the rapidly shifting AI landscape and the critical infrastructure bottlenecks facing organizations building out AI factories and computing clusters.
Spry introduces the pioneered concept of "Deep Networking," a vertically integrated hardware and software solution that embeds telemetry down to the lowest ASIC level while surfacing insights through natural language agentic AI interfaces. He highlights the stark architectural contrast between the cloud era and the AI era: while cloud applications are loosely coupled and resilient to localized infrastructure failures, AI networking is highly tightly coupled, meaning processing speeds automatically drag down to the slowest component.
To achieve "time to first token" efficiency and secure a competitive advantage, Spry urges CIOs and CTOs to steer away from costly, prolonged in-house builds and instead leverage specialized commercial expertise and advanced "Neo Clouds" to optimize hardware investments.
Key Quotes
- "Aria Networks... pioneered this concept of what we like to call as deep networking, and that is this vertically integrated solution that has like deep technology all the way down into the... ASICs, the lowest level of the hardware."
- "In cloud, the applications were loosely coupled to the infrastructure underneath them... In AI networking, that is very much the pendulum has swung the other way. These are very tightly coupled systems."
- "I think that there's going to be more consumers of our solutions and our products that don't have heartbeats... products are going to be consumed more and more by agents."
- "A two percent gain [in networking] can give you outsized impact, you know, like ten to twenty percent more token efficiency. So I think that it's worth steel-manning the counter [instead of driving cost to the lowest component]."
Takeaways
- AI Architecture Demands Tight Coupling: Unlike cloud environments where infrastructure failures easily spin up in alternative VPC regions without user disruption, AI training and inference loops are highly tightly coupled. System performance and model completion rates are dictated by the slowest link, making deep, low-latency networking non-negotiable.
- Optimize via Outsized Technical Gains: When calculating infrastructure ROI, looking strictly for the lowest-priced hardware component is counterintuitive. Investing slightly more in hyper-speed networking can generate a tiny 2% infrastructure optimization that yields a massive 10% to 20% surge in enterprise token efficiency.
- Prepare for Non-Human Users: The traditional SaaS metrics of Daily Active Users (DAU) and Monthly Active Users (MAU) must be refactored to account for AI agents. The industry is moving toward a reality where digital workflows run autonomously 24/7 while human teams sleep, shifting software consumption primarily toward agentic workloads.
- Leverage Forward-Deployed Expertise: The specialized engineering skill set required to stand up GPU data centers is severely lacking in the broader enterprise market. Organizations should avoid the trap of prolonged internal tool builds that delay time-to-market and instead utilize forward-deployed commercial specialists to bootstrap systems rapidly.
Speaker Bio: DJ Spry is Head of Product at Aria Networks. He previously served as Senior Director of Product Management at Juniper, leading the product team following the acquisition of Apstra, where he was an early employee. His commercial networking career includes driving GTM for Open Networking at Dell EMC and serving as a consulting engineer at Juniper. He began his career in the US Air Force and later served as a network engineer and architect for the US Intelligence Community. - Aria website and social handles:
- arianetworks.com
- https://www.linkedin.com/company/aria-networks-inc/
- https://x.com/AriaNetworks
- https://www.youtube.com/@Aria_Networks
Find Amanda Razani on LinkedIn. https://www.linkedin.com/in/amanda-razani-990a7233/
Follow the FTA LinkedIn Page: https://www.linkedin.com/company/full-tech-ahead/
Visit the FTA website: https://fulltechahead.com/
Check out the Substack Channel: https://fulltechahead.substack.com/
Listen
Watch
Mark as Played
Transcript
Episode Transcript
Available transcripts are automatically generated. Complete accuracy is not guaranteed.
SPEAKER_01 (00:19):
Hello and welcome to Full Tech and Head.
I'm your host, Amanda Rizzani.
And with me today, I'm excitedto have DJ Sprite.
He is the head of product forARIA Networks.
How are you doing today?
SPEAKER_00 (00:31):
I'm wonderful.
How are you?
SPEAKER_01 (00:33):
Doing well.
So first, let's talk about ARIANetworks.
Can you share a little bit aboutthe services provided by ARIA
Networks and how you help yourclients?
SPEAKER_00 (00:44):
Yes, uh, ARIA Networks is uh a networking
company that is buildingsolutions specifically for those
that are building out uh AIfactories, AI clusters.
So we have a suite of hardwareand software designed to help
organizations stand up, manage,operationalize, get time to
first token all of the AI thingsthat exist.
(01:05):
So we we have a verticallyintegrated solution.
We've pioneered this concept ofwhat we like to call as deep
networking.
And it is this uh verticallyintegrated solution that um has
like deep technology all the waydown into the, you know, like to
the ASICs, the lowest level ofthe hardware to get telemetry
and be able to perform actionsand closed loop at the like the
(01:27):
very lowest layer of the networkat the at the highest speed
possible, which is critical forthese uh AI workloads.
And then being able to take thatinformation all the way up
through the stack and all theway up to a user interface and a
in an agentec AI product thatthat you know our users will
operate with that have a lot ofthe mon modern conveniences and
(01:48):
and a much more you know naturallanguage interface and have
things like skills and be ableto really take advantage of of
the data that exists at thelower level.
And in you know, like either oneof those, if you would take
either of these two pieces inisolation, then you you have a
very fragmented solution.
And you know, like you if youhave an agentic interface but
you don't have access to all thetelemetry or at the rate you
(02:09):
need or the rate that these AIsystems demand, previous
generations will have uh cannotcollect the telemetry, they're
very slow at it.
And by the time an issue hashappened, you like you don't
even see it.
So if you have an agenticinterface and you don't have
that, then you know you haven'treally moved the ball forward.
And the inverse has is wecollect all the telemetry in the
world.
I mean, one thing is that we'reable to generate more telemetry
(02:33):
and data in these systems thatoutpace our ability to
understand them.
And being able to reason aboutall this data and then the
telemetry, like it allcollectively, that's what we
want to do.
So we provide these solutions,you know, like to our customers
to help them differentiate theirbusiness and get the market
faster.
SPEAKER_01 (02:49):
Well, then you're the right person to speak with
for our topic today, which isthe ever-changing AI landscape
and the infrastructure uh issuesthat business leaders are facing
at the moment.
Yeah, that'd be to get started.
Let's talk about the cost of AIinfrastructure.
It seems to be climbing by theday.
And what I'm hearing a lot is uhthe token use is it not going
(03:12):
very far uh for projects andit's getting more and more
expensive.
So what are you seeing with yourclients?
And what do you have to sayabout this issue?
SPEAKER_00 (03:24):
Yeah, I mean, I think token efficiency is is I
mean, it's a broad term that isan umbrella term and could be
measured in many different ways,but it is a generally agreed
upon concept that allows, youknow, like businesses and
companies to essentiallymeasure, you know, like the
return on investment in one inone dimension, right?
Like there is uh it allows youto say, like, you know, how much
(03:47):
intelligence am I generatingbased on this large capital
expenditure that you know thatthat I've just rolled out?
And it it can't be customized.
And when I say it could bemeasured in many different ways,
is that when we engaged, likethe conversations that we have
with our customers, like somecustomers or neo clouds that we
engage with maybe trying todifferentiate their business and
their portfolio on how much howquickly that they can give you a
(04:09):
response.
Um, so you know, today you youinteract with your, you know,
like LLM of choice, sometimesthe response is delayed versus
if you go to some businesses whoare dealing with like voice as a
service or something, you don'twant that delay.
So you have to deliver, youknow, like tokens or
intelligence very, very quickly.
And others may have specializedmodels like genome or or
something else where they'retrying to measure like their
(04:32):
their token efficiency on adifferent dimension.
And it is like how large of apayload can we del what can we
deploy back?
And and they're so like theyhave this like paradial curve,
this concept where, like,depending on where you are as a
business and what products thatyou're trying to offer, that you
know, like token efficiency is away for you to measure, you
know, like how well that you areserving intelligence to you know
(04:53):
back to your business,particularly when you've, you
know, like you've laid out alarge amount of money within
that way.
It is, it is very important toto track and to be specific
based on like how it is thatyou're trying to differentiate
on on your business, because youdon't want to, you know, like be
can, you know, like you don'twant to track the wrong metric.
And we work with a lot ofcustomers and say, you know,
(05:13):
like where it is it that youwant to be, because there's a
lot of ways that you candifferentiate, like you want to
have some sort of competitiveadvantage of differentiation in
your business.
And so we work with them andidentify like metrics directly
impact or a result of like tokenefficiency in that specific way.
And that's how we work with ourcustomers.
SPEAKER_01 (05:31):
Yeah, absolutely.
So, what are some of the biggestAI bottlenecks or issues that
are facing companies?
And what do you recommend on howthey get past those?
SPEAKER_00 (05:42):
Yeah, I mean, I I I I definitely say that logistics
supply chain is a bottleneckright now.
I mean, there's a lot of, Iwould say, constraints inside in
in the systems for forcompanies.
And depending on where they areand what their business, what
their businesses look like, itcould be everything from like
access to just, you know, likereal estate and power, or it
(06:03):
could be uh access to, you know,the hardware and solutions that
they that you know that they'reactually trying to bring to
market.
Yeah, I think that that's youknow, that's a challenge that
that everyone's having abottleneck and working with
companies and providers thathave access to to the goods and
to the hardware and to softwarethat allow you to get to market
(06:24):
sooner.
I mean it it is I think everyonehave seen it now the pace of
innovation is just likestaggering.
This is every day, you know.
Like I, you know, I open upLinkedIn or X more often than
not because it seems like X iskind of like the heartbeat and
pulse of AI these days.
And like every day is justsomething amazing.
It's hard for me to keep up, andI am in the industry.
It's almost like you have to beunemployed to like keep up with
(06:46):
everything.
And the businesses have to movethat quickly.
So if you want to be in themarket and you want to begin to
take share, then you need tohave access to, you know, like
not only to the capital, but tothe resources that allow you to
like get in production andstart, you know, like monetizing
your business and your assets.
SPEAKER_01 (07:03):
Absolutely.
And it seems like uh manycompanies are struggling with
old legacy technology and thatthe infrastructure they're
trying to use isn't reallyworking.
So where do they start in fixingthis?
SPEAKER_00 (07:15):
It's an interesting question because uh, I mean,
maybe if I take an analogy thatI use, is that every large like
technology shift that we've seenrequired a completely different
mental model or a completelydifferent rewrite of the
software and and how we build,you know, like applications and
and like the infrastructure tosupport those applications.
(07:35):
And if I were to, you know, Icreate a very strong analogy
that I've used before is thatfor like 30 years after the in
invention of like electricalgenerators, we still used steam
engines and we just like kind ofplopped the electrical engine
and kept all of the you know therest of the componentry for the
steam engine in place.
So we didn't see like anyefficiency gains.
And you can kind of see the samething, you know, that happened
(07:57):
with cloud and with theinternet, and um, you know, like
for you know, a while there thatpeople tried to take what they
already know and like apply itto this new world world, you
know, like when cloud, you know,hit the scene, everyone tried
to, you know, have virtualizedinfrastructure and try to run it
into the cloud instead ofbuilding things that are more
cloud native.
And it wasn't until we, youknow, as an industry, I learned
(08:20):
like what are the patterns andhow to really take advantage of
this.
It we like you rewrote the wholestack and you deployed
everything, you know, likecompletely fresh and completely
new.
And so this is kind of ananalogy to that is that you have
to, you know, adopt technologiesthat are you know that are are
are built for this age and andin order to really gain the
advantages out of it.
(08:41):
Like you can't really use thetechnology of of yesteryear
because they were designed fordifferent, you know, like they
are optimized for differentsolutions.
So, you know, like ininfrastructure in cloud, the
applications were very resilientand they were like loosely
coupled to the infrastructureunderneath them.
So applications could die andthey could fell over and get
spun up in some other, you know,like VPC or some other region
(09:03):
inside of your cloud providerdesure and and the end user
wouldn't know it.
In AI networking, that is thatis very much the pendulum that
swung the other way.
These are very tightly coupledsystems.
It generally will drag down tothe lowest common denominator,
meaning by which, like whenyou're training models as an
example, you the modelcompletion is dictated by the
(09:27):
the the like the slowest entitybecause every like all of the
information you know has to goback, you know, like to one
source and then bring back likeall reduce and all shuffle, and
there's these techniques, but itessentially the speed drags to
the slowest, you know, likecomponent.
Or on the inference side, youknow, like you have we, you
know, the the thing that we asconsumers interact with most,
(09:50):
like you type questions andstuff, like those are the
they're very tightly coupledsystems, and you can't operate
in the same way that you wouldin the cloud era.
So you have to have much moreresilient systems that are much
more, you know, like tightlycoupled to the application
space.
Um, so it's a very differentparadigm in that.
And you know, like you see, liketime and time again, and it's
just really businesses need tomove quickly, move as quickly as
(10:12):
they possibly can to thesesolutions for the same reason as
above, right?
Like you want to be able to havea competitive advantage, be it
you know, like speed kills,you're like or in inverse kills
in this market.
So like you really have to havelike a competitive advantage.
The earlier you can adopt it, Ithink the better, like overall
as a business that you would be.
SPEAKER_01 (10:32):
So I've been hearing the term neo clouds thrown
around.
I thought I might ask you, whatis that and why is that an
important part of the AI topicright now?
SPEAKER_00 (10:43):
Yeah, I've been saying, I mean, in simple terms,
I'm not, I'm sure it has a very,you know, like agreed upon
definition, which I can't pullout right now.
But fundamentally, it's just,you know, we have cloud
providers and they host, youknow, compute and databases and
they host resources, you know,like fungible resources that you
could build on top of.
This is think of it as likecloud 2.0, where you're getting
(11:06):
access to GPUs.
And similarly, it's actually aninteresting question.
I mean, it's timed very well tocompare and contrast from like
the last question you asked isit is that Neo Clouds are
building very specifically forthese AI workloads.
And they're they have, you know,either they're building these
GPU clusters on behalf ofcustomers, because it's you
know, like it's it's a very hardand specialized skill to get up
(11:28):
and running, and uh, and how tohost them and and provide, you
know, like interfaces to thecustomers and everything is just
very different from the cloud.
So it is in in one way, is justa people who rent and provide
GPU access, much like that thecloud providers would provide,
you know, databases as anaccess, but it's so specialized
(11:49):
uh and they need to move muchfaster than what the cloud
providers were able to serve orwhat they were like, what really
what their specialty was.
SPEAKER_01 (11:56):
Great.
So AI is advancing very rapidly.
We're seeing a lot of shifts.
When you look into the future,what are some of the trends that
you expect and anticipate?
And how should business leadersprepare for those?
SPEAKER_00 (12:12):
Uh I'm gonna kind of continue the same theme.
I think that the tr the trendthat I expect to see is more.
I did a blog, I'm gonnashameless plug about a blog post
that I wrote, but I mean you askmy question, and it's it's very
similar.
But it is that I think thatthere's going to be more
consumers of our solutions andour products that don't have
(12:33):
heartbeats.
I mean, there's going to bemore, like right now it is very
much humans in loop or humanstaking advantage of these
systems to gain efficiencies.
But I think very, very quicklywe're going to see, you know,
like the if you were to watchmetrics on, you know, they have,
you know, in a SaaS business,they have like DAO and they have
like Mao, monthly active usersand daily active users.
(12:54):
And I think that really thoseconcepts are going to have to
adopt something to like toagents because the more that
these technologies roll out, themore that people get comfortable
with them, the more that youknow that we can accomplish work
while we're sleeping, as anexample.
I mean, I I it's it's prettyawesome.
I I use it, you know, to helpbuild you know our product and I
(13:16):
help to do prototyping.
And I see people that are, youknow, that we work with and
we're doing like two or threethings at one time.
And it's it's just those, youknow, downstream impacts is are
going to be, you know, like whatI expect to see happen is that
yeah, products are going to beconsumed more and more by
agents.
It's probably going to happenfaster than anyone thinks it is,
just because like we talkedabout the rate of changes
happening so quickly.
(13:37):
And it's it's it's kind of likeum, you know, what is that uh
that adage, like very slowly andall at once?
It's like, oh, we're gonna haverobotax leads and it's like,
yeah, whatever.
And then I turn around andthere's like Waymo's and Teslas
everywhere out of Austin.
It's like you turn around andthey're everywhere.
So I think it'll be a bit likethat.
SPEAKER_01 (13:53):
Yeah.
And to that note, when it comesto ROI and expenses, what advice
you have for a CIO or a CTObalancing those AI costs, where
should they focus first?
SPEAKER_00 (14:06):
Uh, I would say it's a good question.
I mean, I will give maybe like aan on an unobvious one.
And that is a lot of a lot ofpeople generally, particularly
in infrastructure, I'll speak oninfrastructure for a second, it
tend to try to get to the lowestprice component or to drive the
cost down.
(14:26):
But when it is that you'retrying to be like this, you
know, term this leading providerof intelligence, that it is
counterintuitive at times thatyou may want to spend more,
right?
Like, and there are certainareas that give you, you know,
like outsized gains, right?
In the in the networking, it's agenerally a small spend compared
(14:46):
to that much larger, you know,like mini zeros bill.
But you know, a 2% gain can giveyou outsized impact, you know,
like 10 to 20% more tokenefficiency.
So I think that it's worth steelmanning the counter.
And that is is that, you know,how much value are you going to
get instead of like what is thelowest cost that I can drive
(15:08):
these things?
And again, I think that that mayhave worked generally well for
previous errors, but but I thinkthat's a counterintuitive that I
think that you know, for me.
The other one is just on thehuman element.
I think that wherever there'spain in the system, or if you
have, you know, like a lot ofresources to allow them, you
(15:29):
know, like allow your team toexp, you know, like experiment,
but there is a you know, and anduse these tools, and there is a
fine line between, you know,like beginning to just burn
tokens on on like flappy bird orsomething.
And so I think certainly, youknow, trying to focus on on
outcomes, like how quickly canwe get to market, or like what
(15:50):
is our token efficiency, or likehow how many sites can we stand
up, or how many prototypes havewe done.
Uh, and so you try to measureand an outcome and see, you
know, like how quickly you canget to that, you, you know, like
using these tools.
And is that way, you know, likeyou, you know, show me an
incentive, I'll show you anoutcome.
SPEAKER_01 (16:09):
So if there was one key takeaway that you could
leave business leaders withtoday in regard to AI
implementation and this new erathat we're in, what would that
be?
SPEAKER_00 (16:18):
Yeah, I I would say in particular in with you know
with ARIA, it is that the youknow AI is here to stay, the cat
is out of the bag, so to speak.
And it's only going to getfaster.
And I and the more that you canengage with companies and the
expertise that can help yourbusiness, um, that you can allow
(16:39):
your workforce to experimentwith very, you know, like very
quickly, particularly on on likethe outcomes for us is
deploying, you know, like howquickly can we stand up, help
our customers stand up, youknow, GPU data centers and
start, you know, time to firsttoken.
So I I would say, you know, tryto engage.
I think many people tend to wantto like paint their idealistic
(17:00):
self, that they can build a lotof these things in-house.
And what you know, the data hasshown time and time and time
again is that generally thesethese internal adoption tools
like don't reap the benefitsthat you would want either
because they drag forever orthey're just not the same as
what you know, like time tomarket is what you would get if
you engage, you know, commercialsolutions or expertise.
(17:23):
And and I think the sooner thatyou can gauge in companies that
allow you to then get theexpertise to understand how
these systems are built, to haveconversations that we have for
deployed engineers.
It's a you know a model thatmany of the AI companies have
because uh quite honestly,there's a it's just a skill set
(17:44):
that is lacking in a in a lot ofthe companies, and even in the
neo clouds or in the enterprisesand et cetera.
And you need somebody to helpyou along the way and that you
can bootstrap, you can leveragesomeone to bootstrap you
quickly.
I I would say try to engage withcompanies that have the
expertise in the domain um as asfast as you can.
It's not that you can'texperiment, but I wouldn't, you
know, like I think that boththings can be true.
(18:05):
I just wouldn't, you know, tryto, you know, pave away yourself
while everyone in the industryis like passing you by.
SPEAKER_01 (18:12):
Yeah, those collaborations and partnerships
are critical.
SPEAKER_00 (18:15):
Yeah.
Yeah, I mean, it helps.
We learn a lot, they learn alot.
SPEAKER_01 (18:19):
Absolutely.
Well, thank you so much forcoming on the show and sharing
your insights with us today.
SPEAKER_00 (18:25):
Absolutely.
It's been a pleasure.
Hope to do it again.
SPEAKER_01 (18:27):
Yes.
And thank you to our audience.
If you have any questions orcomments, please leave them
below and I'll try to respond assoon as possible.
And until next podcast, have awonderful week.
Hello and welcome to Full Tech and Head.
I'm your host, Amanda Rizzani.
And with me today, I'm excitedto have DJ Sprite.
He is the head of product forARIA Networks.
How are you doing today?
SPEAKER_00 (00:31):
I'm wonderful.
How are you?
SPEAKER_01 (00:33):
Doing well.
So first, let's talk about ARIANetworks.
Can you share a little bit aboutthe services provided by ARIA
Networks and how you help yourclients?
SPEAKER_00 (00:44):
Yes, uh, ARIA Networks is uh a networking
company that is buildingsolutions specifically for those
that are building out uh AIfactories, AI clusters.
So we have a suite of hardwareand software designed to help
organizations stand up, manage,operationalize, get time to
first token all of the AI thingsthat exist.
(01:05):
So we we have a verticallyintegrated solution.
We've pioneered this concept ofwhat we like to call as deep
networking.
And it is this uh verticallyintegrated solution that um has
like deep technology all the waydown into the, you know, like to
the ASICs, the lowest level ofthe hardware to get telemetry
and be able to perform actionsand closed loop at the like the
(01:27):
very lowest layer of the networkat the at the highest speed
possible, which is critical forthese uh AI workloads.
And then being able to take thatinformation all the way up
through the stack and all theway up to a user interface and a
in an agentec AI product thatthat you know our users will
operate with that have a lot ofthe mon modern conveniences and
(01:48):
and a much more you know naturallanguage interface and have
things like skills and be ableto really take advantage of of
the data that exists at thelower level.
And in you know, like either oneof those, if you would take
either of these two pieces inisolation, then you you have a
very fragmented solution.
And you know, like you if youhave an agentic interface but
you don't have access to all thetelemetry or at the rate you
(02:09):
need or the rate that these AIsystems demand, previous
generations will have uh cannotcollect the telemetry, they're
very slow at it.
And by the time an issue hashappened, you like you don't
even see it.
So if you have an agenticinterface and you don't have
that, then you know you haven'treally moved the ball forward.
And the inverse has is wecollect all the telemetry in the
world.
I mean, one thing is that we'reable to generate more telemetry
(02:33):
and data in these systems thatoutpace our ability to
understand them.
And being able to reason aboutall this data and then the
telemetry, like it allcollectively, that's what we
want to do.
So we provide these solutions,you know, like to our customers
to help them differentiate theirbusiness and get the market
faster.
SPEAKER_01 (02:49):
Well, then you're the right person to speak with
for our topic today, which isthe ever-changing AI landscape
and the infrastructure uh issuesthat business leaders are facing
at the moment.
Yeah, that'd be to get started.
Let's talk about the cost of AIinfrastructure.
It seems to be climbing by theday.
And what I'm hearing a lot is uhthe token use is it not going
(03:12):
very far uh for projects andit's getting more and more
expensive.
So what are you seeing with yourclients?
And what do you have to sayabout this issue?
SPEAKER_00 (03:24):
Yeah, I mean, I think token efficiency is is I
mean, it's a broad term that isan umbrella term and could be
measured in many different ways,but it is a generally agreed
upon concept that allows, youknow, like businesses and
companies to essentiallymeasure, you know, like the
return on investment in one inone dimension, right?
Like there is uh it allows youto say, like, you know, how much
(03:47):
intelligence am I generatingbased on this large capital
expenditure that you know thatthat I've just rolled out?
And it it can't be customized.
And when I say it could bemeasured in many different ways,
is that when we engaged, likethe conversations that we have
with our customers, like somecustomers or neo clouds that we
engage with maybe trying todifferentiate their business and
their portfolio on how much howquickly that they can give you a
(04:09):
response.
Um, so you know, today you youinteract with your, you know,
like LLM of choice, sometimesthe response is delayed versus
if you go to some businesses whoare dealing with like voice as a
service or something, you don'twant that delay.
So you have to deliver, youknow, like tokens or
intelligence very, very quickly.
And others may have specializedmodels like genome or or
something else where they'retrying to measure like their
(04:32):
their token efficiency on adifferent dimension.
And it is like how large of apayload can we del what can we
deploy back?
And and they're so like theyhave this like paradial curve,
this concept where, like,depending on where you are as a
business and what products thatyou're trying to offer, that you
know, like token efficiency is away for you to measure, you
know, like how well that you areserving intelligence to you know
(04:53):
back to your business,particularly when you've, you
know, like you've laid out alarge amount of money within
that way.
It is, it is very important toto track and to be specific
based on like how it is thatyou're trying to differentiate
on on your business, because youdon't want to, you know, like be
can, you know, like you don'twant to track the wrong metric.
And we work with a lot ofcustomers and say, you know,
(05:13):
like where it is it that youwant to be, because there's a
lot of ways that you candifferentiate, like you want to
have some sort of competitiveadvantage of differentiation in
your business.
And so we work with them andidentify like metrics directly
impact or a result of like tokenefficiency in that specific way.
And that's how we work with ourcustomers.
SPEAKER_01 (05:31):
Yeah, absolutely.
So, what are some of the biggestAI bottlenecks or issues that
are facing companies?
And what do you recommend on howthey get past those?
SPEAKER_00 (05:42):
Yeah, I mean, I I I I definitely say that logistics
supply chain is a bottleneckright now.
I mean, there's a lot of, Iwould say, constraints inside in
in the systems for forcompanies.
And depending on where they areand what their business, what
their businesses look like, itcould be everything from like
access to just, you know, likereal estate and power, or it
(06:03):
could be uh access to, you know,the hardware and solutions that
they that you know that they'reactually trying to bring to
market.
Yeah, I think that that's youknow, that's a challenge that
that everyone's having abottleneck and working with
companies and providers thathave access to to the goods and
to the hardware and to softwarethat allow you to get to market
(06:24):
sooner.
I mean it it is I think everyonehave seen it now the pace of
innovation is just likestaggering.
This is every day, you know.
Like I, you know, I open upLinkedIn or X more often than
not because it seems like X iskind of like the heartbeat and
pulse of AI these days.
And like every day is justsomething amazing.
It's hard for me to keep up, andI am in the industry.
It's almost like you have to beunemployed to like keep up with
(06:46):
everything.
And the businesses have to movethat quickly.
So if you want to be in themarket and you want to begin to
take share, then you need tohave access to, you know, like
not only to the capital, but tothe resources that allow you to
like get in production andstart, you know, like monetizing
your business and your assets.
SPEAKER_01 (07:03):
Absolutely.
And it seems like uh manycompanies are struggling with
old legacy technology and thatthe infrastructure they're
trying to use isn't reallyworking.
So where do they start in fixingthis?
SPEAKER_00 (07:15):
It's an interesting question because uh, I mean,
maybe if I take an analogy thatI use, is that every large like
technology shift that we've seenrequired a completely different
mental model or a completelydifferent rewrite of the
software and and how we build,you know, like applications and
and like the infrastructure tosupport those applications.
(07:35):
And if I were to, you know, Icreate a very strong analogy
that I've used before is thatfor like 30 years after the in
invention of like electricalgenerators, we still used steam
engines and we just like kind ofplopped the electrical engine
and kept all of the you know therest of the componentry for the
steam engine in place.
So we didn't see like anyefficiency gains.
And you can kind of see the samething, you know, that happened
(07:57):
with cloud and with theinternet, and um, you know, like
for you know, a while there thatpeople tried to take what they
already know and like apply itto this new world world, you
know, like when cloud, you know,hit the scene, everyone tried
to, you know, have virtualizedinfrastructure and try to run it
into the cloud instead ofbuilding things that are more
cloud native.
And it wasn't until we, youknow, as an industry, I learned
(08:20):
like what are the patterns andhow to really take advantage of
this.
It we like you rewrote the wholestack and you deployed
everything, you know, likecompletely fresh and completely
new.
And so this is kind of ananalogy to that is that you have
to, you know, adopt technologiesthat are you know that are are
are built for this age and andin order to really gain the
advantages out of it.
(08:41):
Like you can't really use thetechnology of of yesteryear
because they were designed fordifferent, you know, like they
are optimized for differentsolutions.
So, you know, like ininfrastructure in cloud, the
applications were very resilientand they were like loosely
coupled to the infrastructureunderneath them.
So applications could die andthey could fell over and get
spun up in some other, you know,like VPC or some other region
(09:03):
inside of your cloud providerdesure and and the end user
wouldn't know it.
In AI networking, that is thatis very much the pendulum that
swung the other way.
These are very tightly coupledsystems.
It generally will drag down tothe lowest common denominator,
meaning by which, like whenyou're training models as an
example, you the modelcompletion is dictated by the
(09:27):
the the like the slowest entitybecause every like all of the
information you know has to goback, you know, like to one
source and then bring back likeall reduce and all shuffle, and
there's these techniques, but itessentially the speed drags to
the slowest, you know, likecomponent.
Or on the inference side, youknow, like you have we, you
know, the the thing that we asconsumers interact with most,
(09:50):
like you type questions andstuff, like those are the
they're very tightly coupledsystems, and you can't operate
in the same way that you wouldin the cloud era.
So you have to have much moreresilient systems that are much
more, you know, like tightlycoupled to the application
space.
Um, so it's a very differentparadigm in that.
And you know, like you see, liketime and time again, and it's
just really businesses need tomove quickly, move as quickly as
(10:12):
they possibly can to thesesolutions for the same reason as
above, right?
Like you want to be able to havea competitive advantage, be it
you know, like speed kills,you're like or in inverse kills
in this market.
So like you really have to havelike a competitive advantage.
The earlier you can adopt it, Ithink the better, like overall
as a business that you would be.
SPEAKER_01 (10:32):
So I've been hearing the term neo clouds thrown
around.
I thought I might ask you, whatis that and why is that an
important part of the AI topicright now?
SPEAKER_00 (10:43):
Yeah, I've been saying, I mean, in simple terms,
I'm not, I'm sure it has a very,you know, like agreed upon
definition, which I can't pullout right now.
But fundamentally, it's just,you know, we have cloud
providers and they host, youknow, compute and databases and
they host resources, you know,like fungible resources that you
could build on top of.
This is think of it as likecloud 2.0, where you're getting
(11:06):
access to GPUs.
And similarly, it's actually aninteresting question.
I mean, it's timed very well tocompare and contrast from like
the last question you asked isit is that Neo Clouds are
building very specifically forthese AI workloads.
And they're they have, you know,either they're building these
GPU clusters on behalf ofcustomers, because it's you
know, like it's it's a very hardand specialized skill to get up
(11:28):
and running, and uh, and how tohost them and and provide, you
know, like interfaces to thecustomers and everything is just
very different from the cloud.
So it is in in one way, is justa people who rent and provide
GPU access, much like that thecloud providers would provide,
you know, databases as anaccess, but it's so specialized
(11:49):
uh and they need to move muchfaster than what the cloud
providers were able to serve orwhat they were like, what really
what their specialty was.
SPEAKER_01 (11:56):
Great.
So AI is advancing very rapidly.
We're seeing a lot of shifts.
When you look into the future,what are some of the trends that
you expect and anticipate?
And how should business leadersprepare for those?
SPEAKER_00 (12:12):
Uh I'm gonna kind of continue the same theme.
I think that the tr the trendthat I expect to see is more.
I did a blog, I'm gonnashameless plug about a blog post
that I wrote, but I mean you askmy question, and it's it's very
similar.
But it is that I think thatthere's going to be more
consumers of our solutions andour products that don't have
(12:33):
heartbeats.
I mean, there's going to bemore, like right now it is very
much humans in loop or humanstaking advantage of these
systems to gain efficiencies.
But I think very, very quicklywe're going to see, you know,
like the if you were to watchmetrics on, you know, they have,
you know, in a SaaS business,they have like DAO and they have
like Mao, monthly active usersand daily active users.
(12:54):
And I think that really thoseconcepts are going to have to
adopt something to like toagents because the more that
these technologies roll out, themore that people get comfortable
with them, the more that youknow that we can accomplish work
while we're sleeping, as anexample.
I mean, I I it's it's prettyawesome.
I I use it, you know, to helpbuild you know our product and I
(13:16):
help to do prototyping.
And I see people that are, youknow, that we work with and
we're doing like two or threethings at one time.
And it's it's just those, youknow, downstream impacts is are
going to be, you know, like whatI expect to see happen is that
yeah, products are going to beconsumed more and more by
agents.
It's probably going to happenfaster than anyone thinks it is,
just because like we talkedabout the rate of changes
happening so quickly.
(13:37):
And it's it's it's kind of likeum, you know, what is that uh
that adage, like very slowly andall at once?
It's like, oh, we're gonna haverobotax leads and it's like,
yeah, whatever.
And then I turn around andthere's like Waymo's and Teslas
everywhere out of Austin.
It's like you turn around andthey're everywhere.
So I think it'll be a bit likethat.
SPEAKER_01 (13:53):
Yeah.
And to that note, when it comesto ROI and expenses, what advice
you have for a CIO or a CTObalancing those AI costs, where
should they focus first?
SPEAKER_00 (14:06):
Uh, I would say it's a good question.
I mean, I will give maybe like aan on an unobvious one.
And that is a lot of a lot ofpeople generally, particularly
in infrastructure, I'll speak oninfrastructure for a second, it
tend to try to get to the lowestprice component or to drive the
cost down.
(14:26):
But when it is that you'retrying to be like this, you
know, term this leading providerof intelligence, that it is
counterintuitive at times thatyou may want to spend more,
right?
Like, and there are certainareas that give you, you know,
like outsized gains, right?
In the in the networking, it's agenerally a small spend compared
(14:46):
to that much larger, you know,like mini zeros bill.
But you know, a 2% gain can giveyou outsized impact, you know,
like 10 to 20% more tokenefficiency.
So I think that it's worth steelmanning the counter.
And that is is that, you know,how much value are you going to
get instead of like what is thelowest cost that I can drive
(15:08):
these things?
And again, I think that that mayhave worked generally well for
previous errors, but but I thinkthat's a counterintuitive that I
think that you know, for me.
The other one is just on thehuman element.
I think that wherever there'spain in the system, or if you
have, you know, like a lot ofresources to allow them, you
(15:29):
know, like allow your team toexp, you know, like experiment,
but there is a you know, and anduse these tools, and there is a
fine line between, you know,like beginning to just burn
tokens on on like flappy bird orsomething.
And so I think certainly, youknow, trying to focus on on
outcomes, like how quickly canwe get to market, or like what
(15:50):
is our token efficiency, or likehow how many sites can we stand
up, or how many prototypes havewe done.
Uh, and so you try to measureand an outcome and see, you
know, like how quickly you canget to that, you, you know, like
using these tools.
And is that way, you know, likeyou, you know, show me an
incentive, I'll show you anoutcome.
SPEAKER_01 (16:09):
So if there was one key takeaway that you could
leave business leaders withtoday in regard to AI
implementation and this new erathat we're in, what would that
be?
SPEAKER_00 (16:18):
Yeah, I I would say in particular in with you know
with ARIA, it is that the youknow AI is here to stay, the cat
is out of the bag, so to speak.
And it's only going to getfaster.
And I and the more that you canengage with companies and the
expertise that can help yourbusiness, um, that you can allow
(16:39):
your workforce to experimentwith very, you know, like very
quickly, particularly on on likethe outcomes for us is
deploying, you know, like howquickly can we stand up, help
our customers stand up, youknow, GPU data centers and
start, you know, time to firsttoken.
So I I would say, you know, tryto engage.
I think many people tend to wantto like paint their idealistic
(17:00):
self, that they can build a lotof these things in-house.
And what you know, the data hasshown time and time and time
again is that generally thesethese internal adoption tools
like don't reap the benefitsthat you would want either
because they drag forever orthey're just not the same as
what you know, like time tomarket is what you would get if
you engage, you know, commercialsolutions or expertise.
(17:23):
And and I think the sooner thatyou can gauge in companies that
allow you to then get theexpertise to understand how
these systems are built, to haveconversations that we have for
deployed engineers.
It's a you know a model thatmany of the AI companies have
because uh quite honestly,there's a it's just a skill set
(17:44):
that is lacking in a in a lot ofthe companies, and even in the
neo clouds or in the enterprisesand et cetera.
And you need somebody to helpyou along the way and that you
can bootstrap, you can leveragesomeone to bootstrap you
quickly.
I I would say try to engage withcompanies that have the
expertise in the domain um as asfast as you can.
It's not that you can'texperiment, but I wouldn't, you
know, like I think that boththings can be true.
(18:05):
I just wouldn't, you know, tryto, you know, pave away yourself
while everyone in the industryis like passing you by.
SPEAKER_01 (18:12):
Yeah, those collaborations and partnerships
are critical.
SPEAKER_00 (18:15):
Yeah.
Yeah, I mean, it helps.
We learn a lot, they learn alot.
SPEAKER_01 (18:19):
Absolutely.
Well, thank you so much forcoming on the show and sharing
your insights with us today.
SPEAKER_00 (18:25):
Absolutely.
It's been a pleasure.
Hope to do it again.
SPEAKER_01 (18:27):
Yes.
And thank you to our audience.
If you have any questions orcomments, please leave them
below and I'll try to respond assoon as possible.
And until next podcast, have awonderful week.
Popular Podcasts
Stuff You Should Know
If you've ever wanted to know about champagne, satanism, the Stonewall Uprising, chaos theory, LSD, El Nino, true crime and Rosa Parks, then look no further. Josh and Chuck have you covered.
Betrayal Weekly
Betrayal Weekly is back for a new season. Every Thursday, Betrayal Weekly shares first-hand accounts of broken trust, shocking deceptions, and the trail of destruction they leave behind. Hosted by Andrea Gunning, this weekly ongoing series digs into real-life stories of betrayal and the aftermath. From stories of double lives to dark discoveries, these are cautionary tales and accounts of resilience against all odds. From the producers of the critically acclaimed Betrayal series, Betrayal Weekly drops new episodes every Thursday. If you would like to share your story, you can reach out to the Betrayal Team by emailing them at betrayalpod@gmail.com and follow us on Instagram at @betrayalpod and @glasspodcasts. Please join our Substack for additional exclusive content, curated book recommendations, and community discussions. Sign up FREE by clicking this link Beyond Betrayal Substack. Join our community dedicated to truth, resilience, and healing. Your voice matters! Be a part of our Betrayal journey on Substack.
Dateline NBC
Current and classic episodes, featuring compelling true-crime mysteries, powerful documentaries and in-depth investigations. Follow now to get the latest episodes of Dateline NBC completely free, or subscribe to Dateline Premium for ad-free listening and exclusive bonus content: DatelinePremium.com