August 22, 2024 • 29 mins
The power of edge AI transforms the embedded systems industry with new levels of performance, energy efficiency, memory optimization, and accuracy. But how can businesses leverage these capabilities to build applications that deliver real value?
In this episode, we explore where and how the latest technology is deployed and how companies should approach their next-generation solutions. We dive into real-world edge AI applications, highlight their impact on various industries, and showcase their potential to solve complex challenges.
Join us as we explore these ideas with:
Alex Wood, Global Marketing Director, Avnet
Christina Cardoza, Editorial Director, insight.tech
Brandon Lewis, insight.tech contributor
Alex answers our questions about:
- Current state of the embedded systems industry
- Driving factors behind edge AI applications
- Real-world use cases from customers
- When to leverage the latest technologies
- Meeting the different technology demands
- Processor advancements and benefits
Related Content
To learn more about the latest edge AI innovations, see what Intel partners across the global do in their industries. For the latest innovations from Tria Technologies, follow them on LinkedIn.
Mark as Played
Transcript
Episode Transcript
Available transcripts are automatically generated. Complete accuracy is not guaranteed.
(00:00):
(upbeat music)
- Hello, and welcometo "insight.tech Talk,"
where we explore the latest IoT, edge, AI,
and network technologytrends and innovations.
(00:20):
I'm your host, Christina Cardoza,
Editorial Director of insight.tech.
And today we're going to be exploring
the embedded systems industrywith two special guests.
First, we have Brandon Lewis,
longtime friend andcontributor of insight.tech,
who will be guest hostingthe podcast today.
And joining him is Alex Wood,
Global Marketing Director of Avnet.
But as always, before we get started,
let's get to know our guests.
(00:42):
Brandon, I'll start with you.
What can you tell us aboutwhat you've been up to
these days?
- Sure, so I've been doinga lot of coverage still
of the embedded and IoT space.
I've been getting a lot closerwith developers recently,
seeing what they're workingwith, what they're working on,
from a tools and chipsets perspective,
which is always a lot of fun
getting closer and closer to the action.
(01:02):
- Yeah, absolutely.
And looking forward to theconversation today with Alex.
Alex, welcome to the podcast.
What can you tell usabout what you do at Avnet
and the company itself?
- At Avnet, we've just launched
our new compute brand, Tria,
which is born out of theold Avnet Embedded business,
which used to be Avnet Integrated,
which used to be MSC Technologies.
So it's a bit of an evolutionover the last few years,
(01:26):
but we've got a reallygood, strong brand now.
So I've kind of been focusing entirely
on launching that new brandfor the last six months.
It's nice to take a step back
and have a cup of coffee with you
and talk a little bitmore about technology
and less about branding guidelines.
- Absolutely.
Before I throw it over to Brandon, Alex,
I wanted to start the conversation,
especially since you havethis new product coming out.
(01:47):
What are the technology trends?
What's going on in this space
that you guys launched a new line,
or especially with edge and AI
becoming more prevalent in the industry?
What are some thingsthat you see going on?
- I think we're at a nexuspoint, really, in the industry.
With AI, there's a lot of emphasis
on putting things into the cloud,
(02:08):
and there's a lot of pushback
from people that want to putthings on the edge as well.
So you've got one half going to the cloud,
the other half going to the edge,
and both of them have their own challenges
and potential setbacks.
So that's really whatwe're seeing at the moment
is customers are saying,"We want to leverage this,
"but we're not entirely surehow we can leverage this."
And it really is a sort of,
there's no perfect silver bullet.
(02:30):
So we've got to find theright path for our customers.
- It's interesting you bring that up
because obviously a lot of people
who are going to the cloud
are really looking forthings like more performance,
working with bigger datasets usually.
And on the edge, you tend to think
that you have a need for lowerpower, quick inferencing.
(02:50):
But then we see all theseGPUs and stuff coming out,
these huge, powerful GPUs.
And I wonder, number one, whatare some of the applications
that are driving thingsat the edge, you see?
And then do they really need,
do they need the performance of a GPU?
Can they get away with something else?
Like, are you seeing morelow power at the edge,
(03:11):
more performance requirements?
What is it?
- Yeah, I think poweris the key thing, right?
That's going to be themake or break for AI.
At the moment, AI is super power hungry.
It's consuming a vast amount of data.
It's really putting Bitcoin,
it's making Bitcoin look almostpower efficient right now.
With the amount of power it's consuming.
(03:33):
And I think for a lot of businesses,
people don't realize how muchpower AI applications consume
because they don't see it.
They've sort of outsourced the demand.
Like you run an AIapplication at the edge,
it's hugely power hungry.
And you have to deal with that problem,
the power and the heat at the edge.
(03:55):
If you're sending it off to a data center,
you don't see the challengesthat it brings up.
So it's easy for peopleto forget about that.
So I think reducing the power requirements
of performing these applications
is going to be a key challenge.
And that's going to make or break
whether or not AI sticksaround in this hype cycle,
depending on how youdefine AI and how it works.
And accessing those large data models,
(04:15):
being able to process things
and also absorb data andprocesses in real time.
The applications allrequire more efficient,
more energy efficient, moreheat efficient processing.
And I think that's goingto be the challenge.
- You're a marketing guy.
By the way, thanks for the rebrand.
- Sorry. (laughs)
- No, no, no.
I was going to say, I love Avnet.
(04:38):
It was sometimes confusing
which Avnet, I was referring to, right?
So I think the rebrandwas great with Tria.
So this push of a lot of these super,
what we would consider embedded,
like super big or superhigh performance processors,
based on what you said, is this marketing?
(05:00):
Are we just marketing toget, to sell more units?
Do we really need that?
And what are some usecases that you're seeing
in real world use cases?
We all hear about computervision and stuff like that,
but what's the realitylike from your customers?
- There is that speeds andfeeds elements of marketing.
So it can perform an extra amount of tops.
(05:24):
It can clock at this frequency.
It's got even more RAM.
And if I'm building my gaming PC,
then that's a sort oflike, "Oh, this is great.
I want to be able to get thisextra amount of frame rates.
I want to be able torender videos much faster."
But at the same time, you then have to,
like upgrading graphicscard at my last upgrade,
I had to get a PSU that was twice the size
(05:46):
of the previous PSU.
And you're just like, "Wow, I'mpushing a thousand watts now
to run a proper PC rig"
when it used to be like300 watts was a lot.
That's triple the amount.
For customers, that's the issue.
We had the energy crisis recently.
That brought it to the top of the agenda.
(06:06):
And now it's eased offa little bit for now,
but it's not so long before I think
it's going to come back up again.
The energy consumption, thepower is going to be critical.
So it's not so much about gettinga more powerful processor,
the most powerful processor.
It's about balancing consumption,longevity, capability,
(06:28):
specific to the application.
For customers like that, okay,there is a marketing element.
You want to buy theabsolute top of the range,
the flagship processor,
when actually you might not need that.
But sometimes you do.
And it depends on the application
for what you're going to do.
I'm the least marketing-ymarketing-y guy in that respect.
I'm kind of like, I'd rathersit down with the customer
(06:49):
and say, "Okay, tell me whatyou're actually building."
Rather than just say, "Yes,you need the top of the range.
"You need the i9 immediately."
- What are they building?
What have you seen?
- There's loads of different things
that we're working withcustomers on at the moment.
And a lot of the applications, I mean,
there's a crazy amount.
Everything from new farming applications.
(07:10):
I was reading about anopportunity mentioning no names.
I mean, there's a lot ofarticles at the moment
about more efficient farming
and artificial intelligence being used
as an alternative to thingslike dangerous forever chemicals
that are being put into the soil.
So can you train an AIrobot to move around fields
(07:31):
and identify weeds,being able to tell weeds
and pests apart from cropsand non-harmful animals,
and to be able to organize accordingly.
One of my friends worksin the farming industry
here in the UK,
and he works in a farmingmanagement industry.
So crop checking.
And he has to walk through fields,
(07:53):
taking photos of the different plants
and then educate peopleworking in the fields
to tell the difference betweenthe different varieties
of the plant and whichone to select for breeding
to build the best crop.
And you can create an AIapplication in the field
that does that for you.
You don't necessarily wantto put all of that content
into a data center.
You want to be able toprogram the robot at the edge
(08:15):
to be able to do that.
So we're seeing applicationslike that in agriculture.
And those are edge-based applications.
You don't necessarily have areliable cell data connection
all of the time.
You want to be able to do thatedge-based AI recognition.
And then at the oppositeend of that spectrum,
so you've got the massiveindustrial agriculture use case,
(08:36):
and then we've got automaticlawnmowers for people at home
and being able to map thebest path around the lawn,
but then also being able to spot hazards
and deal with hazardsaround the lawn as well.
So one is a sort of greatfuture-facing altruistic solution.
The other one is a morepractical real life solution,
but it's those practicalchallenges in the real world
(08:58):
that really put thetechnology to the test.
- Are both of those visionapplications, I'm assuming?
Like camera vision?
- Yeah, yeah.
So both of those, I mean, both customers,
one is vision, one can bemore radar sensor application,
but vision is where the jump is
(09:18):
in terms of the processing requirements.
So that live vision AI,
so being able to understandwhat it's looking at
in as quickly as possible,identify it reliably
and act on that identification
instead of having to send signals back
to a data center for crunchingand then get it back again.
It's being able to do thatin a short amount of space
and a short amount of time.
- So this is exactlywhere it's like, okay,
(09:40):
well, you got your trade off time.
It's like decision time, right?
Beause now you're saying, all right,
well, you've got vision out there
and these are probablyboth mobile, I'm assuming,
or semi-mobile, right?
And you have to send, at least
in the industrial ag use case,
you're sending that back somewhere, right?
(10:00):
So is this the place where you're like,
how many GPU execution units can I fit
into this or are you really,
with Tria now, are youtaking it case by case
and saying, look, I mean,from a cost perspective,
let's figure out form,fit and function here.
(10:21):
And it's not top of the line.
Is that the case?
- A lot of customers willhave several different tiers
of the product that they're creating.
So especially for different markets
where there's a different appetite
and also different sizesof the amount of things
that they need to crunch.
So for agriculture, you'llsee that there's the top
of the range where they wantto have mass scale farming,
(10:44):
say in America's with the giant fields
and they want to be ableto do things at speed.
They'll have the topof the range solution.
You buy something really big,it will work in the field.
It's going to cover ahuge amount of distance
in a huge amount of time for a giant farm.
So they have the money,
they have the ability to invest in that.
And then you'll want tohave a slightly slower,
slightly cheaper mid-rangeapplication as well.
(11:06):
And then you want the lower endrange as well for the market
and then let the consumer decide.
Obviously you want to sellthem the best solution,
but sometimes it's notgoing to be an option.
And it's balancing,most customers will have
various different levels of capability
and sell that to the end userbased on their application.
(11:28):
And for me, that's where theindustry is driven forward
by the actual application andwhether or not the end user
feels the need for that amount of use.
I'm always reminded of the picture
that does the rounds onthe internet of the field
and the path that leads around the corner,
(11:49):
like an L-shaped corner.
And then there's a troddenpath across the field
where people have justwalked across diagonally
and it's like designversus user experience.
And I think that likethe last cycle of AI,
there was all of thissort of exciting talk
about what was possible,but at the end of the day,
(12:09):
what was successful and wasn't successful
was defined by people actually using it
and finding it useful.
So the applications that were created,
some of them stuck around,some of them didn't.
It was the same with blockchain
when blockchain wasskyrocketing in usefulness
and the same with NFTs,Bitcoin, that kind of thing.
People actually finding ituseful as an application
(12:30):
and being able to use it every day
and decided what stuckaround and what didn't.
- The same thing seems to havehappened in the IoT space.
There were a bunch of different use cases
that were really pushedhard, like smart home stuff.
And there's a point atwhich as a consumer,
not just like a B2C consumer,but any kind of consumer
(12:51):
where you just eitherdon't need any more of that
or it's just not really practical.
It was a great proof of concept,
but it's not useful at thescale that it's being promoted.
And I think we run into thatdanger zone here with AI too,
where it's like there's alot of vision type stuff
(13:11):
that's getting pushed and it's cool.
And I know that themargins are bigger there,
but ultimately, a lot ofthe actual deployments
aren't going to be exactlywhat you see out in the media.
And it sounds like you're talking about
with the trodden pathacross the fields, right?
It's like the use cases,the demand in the market
(13:32):
is going to start driving exactlywhere this technology goes
and then how it evolves.
- Yeah, you knew that the IoT concept
had reached the top of its hype cycle
when there was IoT toasters on the market.
And okay, like we were saying before,
there's different tiers ofthe products that's available.
Some people will go for that top tier
(13:53):
and some people will just be like,
I want my toast to beslightly more toasted.
I just turn a knob on it, sameas I did back in the 1950s.
It doesn't need to beany more smart than that.
I do like, I recentlyupgraded my aging fridge
to a semi-IoT fridge thattells me if the door's open
or if the temperature'stoo high or too low.
(14:16):
And for me, like I don't need a fridge
with a screen on the front
that gives me informationabout the weather
because I've got a separatedisplay in my kitchen.
I don't need somethingwhere you knock on the door
and it shows me the products behind it.
I don't need a camera in there,
but I do like it if it warns me
if the door's been left openand it beeps on my phone.
(14:37):
And that's usually because mypartner's been loading food
into the fridge andforgot to close the door.
And then I'm in here in my room
and I'm just like, youleft the fridge door open.
Those real life applicationsare what sticks around.
So IoT is now quite a mature market
where the businesses that are investing
in that level of technology,
they put all of thetechnology into the device.
(14:59):
The consumer demand forthat sort of technology
cools off a little bit to a level
where the consumers understandwhat's beneficial to them
in their everyday life.
We've got another customerthat we're working with
that makes industrial cookers.
So for cooking consistently huge amounts
(15:20):
of the same identical foodstuffs
over and over and over again.
There's an IT model, an IoT model there
because you want to be able to control
all of the different ovens
and also manage a hundreddifferent ovens at the same time
and know if one of themis over temperature
or under temperature, that kind of thing.
There's applications therethat we're working with
where that is a requirement
(15:41):
where it might not have been 50 years ago
when cookers were being used
as an industrial scale like that.
- So unfortunately whathappens with this hype cycle
like you mentioned is thateverybody has these huge ideas,
these grandiose visions of what the future
is going to be like, wherewith IoT for example,
it was everything is going to be connected
(16:01):
and your toast is goingto be ready in the morning
and your car is going to be sitting there
waiting to drive you off to work
and it's going to beperfectly climate controlled
and by the time themarket starts to mature
and people realize it's going to cost you
a quarter of a milliondollars per consumer
to realize that vision andit's not going to happen,
everyone experiences a sort of letdown,
(16:23):
that's the trough ofdisillusionment, right?
But that doesn't mean that thetechnology is actually dead
or even unsuccessful, right?
It just means that it'sevolved in some different way
and I think what you'redescribing with AI here
and even that last industrialovens sort of example
is like, hey, there are alot of use cases out there
(16:45):
that aren't necessarily thebiggest, baddest processor,
RAM combination that youcould potentially have
but the volume's there and it exists.
- That connects up withwhat we were talking about
with power efficiency, so understanding
you get all of theinnovation, the excitement,
all the things we couldadd and then you say,
(17:07):
yeah, but I need realistically,practically to run it
with this amount of power drawin order to get what I want.
So you got to sacrifice something
in order to get something else.
Sort of like with an electric car,
you add loads of bells and whistles to it,
it gets heavier and heavier
to the point that the range drops
and then you're, well, Iwant a long range model
so I've got to increase the aerodynamics
(17:28):
which means making it looka little bit less attractive
and strip out things like power seats
in order to reduce the weight as well.
So you've got to find that middle space,
that sweet spot in thesesorts of applications.
- How does the portfolio, Tria,
like expand or develop inorder to meet that range,
(17:50):
that range of requirement?
- I think we've got a pretty good range
that goes from tiny littlelow power compute applications
all the way up to the COM-HPCs
with the server gradeIntel processors in them.
So, and like the COM-HPCswith the Intel processors
(18:10):
are designed foredge-based image processing
and AI applications, butthey're larger as well.
So you have to have a balance between size
and power consumption andwhat they're capable of.
So a lot of the larger,the COM-HPC modules
(18:31):
are this sort of size, they'resort of motherboard sized
which means that you've got to put them
inside a dedicated case.
You couldn't just embedthem directly into a product
unless it was a really big product.
So for things like edge securityor public transportation,
so AI applications andpublic transportation
is another thing that we'reworking on at the moment.
Being able to take data froma huge number of sensors
(18:54):
from a train or othervehicle or train station,
analyze them all, reactto them in real time.
That pretty much requiresan on location server
because sometimes you can'trely on the data network
being reliable.
And that means that we're using those
for those sorts of applicationsin standalone servers.
But a lot of the requirements,
(19:14):
we've got ones for industrial automation.
So again, we're workingwith Intel on cobotics
with one of our customers,building real time image sensors
into a cobotics, a cooperativerobotics environment.
So a robot can operate in thesame space as a human safely.
So if the human moves into that space,
(19:36):
the robot arm stopsmoving, can move around.
If the human picks something up,
the robot knows where it isand can take it off them again.
We were demonstratingan early example of that
at Embedded World in Nuremberg this year.
That was built around a combination
of the Intel based ComExpressmodules that we have
and the Intel based, actually no,
(19:56):
it's Intel based SMARC modules.
And then our Intel based COM-HPC modules
for the image processing.
And those two thingscommunicating with each other.
So getting the signalsfrom the cameras analyzed
and then communicating withthe robot in real time as well.
So there is that sort of,
how useful is the environmentthat you're creating,
the application that you're creating there
(20:17):
versus the amount of power,
the amount of processing that you need,
the amount of space you needin that environment as well.
For some customers, that's a pinnacle.
So it's giving them the option to say,
okay, well, I need cobotics,
I need to have a reliable environment
and therefore I need thatextra processing power
and the associated costs thatcome with setting that up
(20:37):
and developing and installing it.
Whereas other manufacturers,
they might want to just havean enclosed robotic space,
no cobotics required.
You got to, like you were saying before,
you have to create thepotential for innovation.
So you have to inspire the customers
with that new technology,that new possibility,
and then let the customer
(20:58):
then build that application around it.
And if it works for them,
then that creates the foothold
for that technology to develop further.
And for sometimes you'llcreate that new technology
like a lot of the AI applications
that we're seeing at the moment
where the customer, theuser can't really find
that sort of really, that killer app point
(21:19):
that becomes a foothold forthe technologies to develop.
- Tria has almost used the bad A word now,
the old A word.
(laughs)
Within the Tria Portfolio,
obviously it's pretty expansive, right?
I mean, there's lots of options.
What is the Intel portfolio look like?
I mean, are you offering Atom, Core, Xeon,
(21:39):
you know, the sort of the gamut
or what does that looklike in terms of scale?
- Yeah, pretty much the full gamut.
I think within the mobile processor space,
like I said, up to the COM-HPC level,
we can put server gradeprocessors onto those.
But at that point you may aswell have an actual server.
(22:01):
So it depends on the size, the shape
that you need to put it into.
So yeah, we typically offer the Atom
and the Core series and theXeon series at the server end.
We have those, it's really cool to see
what the product team does,
putting things into such a small space.
I've been working with motherboards
(22:22):
and processes for motherboards
for years and years and years.
So to see that sort ofcomputing application
in such a small packagewith heat management,
thermal management is a fine art.
And watching the team developthose sorts of applications
in the environment that theproduct's going to be used in
is a fascinating challenge.
(22:43):
So being able to deploylike an Intel processor
and its capabilities andthe new AI based processes
we're working on as well,
to bake those into a small product
to be able to use at theedge is pretty exciting.
- Well, cool.
I mean, it's really exciting to see
more of the AI in action thanAI in advertisement, right?
(23:05):
So really looking forward toseeing how this continues.
- Brandon I actually wanted to ask you
because you covered embeddedworld for us this year,
which feels like it was lastyear at this point already,
but there was a lot of nextgeneration edge processors
that came out that Intel launched
Intel new core processorscalled Ultra Intel® Arc™ GPU.
(23:27):
So I'm curious, what have you been seeing
around the industry, especiallyas we've talked about
all these use cases and constraints
of how the latest processorsand technology advancements
are helping some of thepartners in this space?
- There's obviously like the software side
and the silicon side,on the software side,
you've got DevCloud and OpenVINO™
has got a really good foothold,
(23:48):
really helping streamline and accelerate
the development of models.
And there's even Intel® Geti™
which is even furtherback on the training side
and just making it easier there.
On the Silicon side, man, the core Ultras,
like the AI PCs, I think thatthey're a really nice fit
(24:09):
in this sort of spectrumthat Alex is talking about
because they enable youto scale up and scale down
even with inside the same skew, right?
Because you've got alot of different compute
that's available to you.
These heterogeneousprocessors where you can say,
look, I want a performance core,
I want an efficiency corefrom a CPU standpoint,
(24:31):
but then also, they've gotgraphics execution units built,
integrated GPUs where youcan do acceleration there.
And then you bring in neural accelerators.
So, you can get this sort of ability
to move your application inone direction or the other
based on what is availableon the SoC or chip set.
(24:53):
And that just givesyou so much flexibility
because at that point toAlex's point about efficiency
and power consumption,you're using the right core
for the right workload, right?
And that's really ultimatelywhat it's all about
because that allows something
that would traditionallyhave been a smaller
or less expensive processorto accomplish more.
(25:17):
And really that's kind ofthe name of the game here.
- That's a really good point, Brandon.
I was at Intel's AI event recently.
They had that globalevent where they showcased
all of their latest AI technologies.
The applications there tolook at some of the partners
that we're showcasing were fascinating
for how you can take AI to accelerate
(25:39):
an application at the edge.
There were things likesupermarket checkout applications
which were automaticcheckouts that recognize
what it is you're holdingand queue management,
automating supermarket management as well.
But it was really coolto see the applications
that Intel was developing at the Olympics.
(26:00):
So, the athlete applicationsthat they developed there,
that's a really great wayof taking the technology
and showing a real life usecase to capture the imagination
of potential developers of the technology.
And the case study video that they showed
of the technology being used in Africa
(26:21):
to sort of scout a hugenumber of potential athletes
and then find potential future Olympians
based on image processingusing that platform.
That was a really cool, thatreally captured my imagination.
It really stuck with me.
But being able to take thatAI processing to the edge
(26:41):
and in a laptop as well,
it goes back to what we weresaying at the beginning,
taking that high power today, it's hugely,
they're large units,they're very powerful,
compressing it, making it smaller,
making it more energy efficient,
being able to put an AIapplication into a laptop,
a laptop-sized device thatcan be used in the field
(27:02):
is really exciting.
I think it was Dell that was up on stage
that was showing the laptops
that they're going to be releasing
with built-in AI applications.
So it's an AI device insteadof a computing device
and really leaning into that collaborative
AI application environment.
You've got a greatexample from the Olympics
(27:23):
that Intel's done, but it's a blank slate.
I'm really excited tosee what developers do
with that amount of AI processingtechnology at the edge,
instead of having todepend on sending stuff
back to a huge data center and back again.
And I think that's going tobe a turning point really
for the future for AI at the edge.
(27:43):
- Honestly, I think a lot'sgoing to be about sustainability
and something I forgot to bring up was,
man, have you ever put afarmer's market piece of produce
next to a supermarket piece of produce?
It's weird.
- I don't know where you'regoing with that, Brandon.
- Well, you were talking aboutnot using chemicals, right?
(28:04):
Not having to use as many chemicals.
And when you put the farmer'smarket versus the supermarket
like something is not right here.
But sustainability in the future,
I think is really important
and I think all the thingsthat you've been talking about
and we've discussed todaywill help us on that path.
- Yeah, for sure.
- It's amazing, allthe different use cases
(28:25):
and everywhere you can gowith these AI applications.
I can't wait to see where else we go,
especially with partners like Avnet.
So it's been a great conversation, guys.
Thank you for joining.
Before we go, Alex, I justwant to throw it back to you
one last time, if there'sany final thoughts
or key takeaways you wantto leave with us today.
- Like I said at the beginning
and like we were kind of leadingback into at the end there,
(28:45):
I think that AI is at anexus point at the moment
and I think edge computingis a nexus point as well.
So that's advancement inedge-based AI applications.
So being able to takeit away from the cloud
and onto the device,that's the nexus point.
If you're watching this,find those applications
and tell us about them if you've got them.
I think it's a reallyexciting time to be working
(29:07):
in computing on a small formfactor with AI in this space.
- Yeah, and I invite all of our listeners
to visit the Avnet website,
check out their new product line,
see how they can help youtake some of your AI efforts
and initiatives off the ground.
So thank you both again, Brandon,
it's always great connecting with you.
You've always been ourembedded systems expert
and thanks to our listenersand thanks Alex from Avnet.
(29:31):
Until next time, this hasbeen "insight.tech Talk."
(upbeat music)
(upbeat music)
- Hello, and welcometo "insight.tech Talk,"
where we explore the latest IoT, edge, AI,
and network technologytrends and innovations.
(00:20):
I'm your host, Christina Cardoza,
Editorial Director of insight.tech.
And today we're going to be exploring
the embedded systems industrywith two special guests.
First, we have Brandon Lewis,
longtime friend andcontributor of insight.tech,
who will be guest hostingthe podcast today.
And joining him is Alex Wood,
Global Marketing Director of Avnet.
But as always, before we get started,
let's get to know our guests.
(00:42):
Brandon, I'll start with you.
What can you tell us aboutwhat you've been up to
these days?
- Sure, so I've been doinga lot of coverage still
of the embedded and IoT space.
I've been getting a lot closerwith developers recently,
seeing what they're workingwith, what they're working on,
from a tools and chipsets perspective,
which is always a lot of fun
getting closer and closer to the action.
(01:02):
- Yeah, absolutely.
And looking forward to theconversation today with Alex.
Alex, welcome to the podcast.
What can you tell usabout what you do at Avnet
and the company itself?
- At Avnet, we've just launched
our new compute brand, Tria,
which is born out of theold Avnet Embedded business,
which used to be Avnet Integrated,
which used to be MSC Technologies.
So it's a bit of an evolutionover the last few years,
(01:26):
but we've got a reallygood, strong brand now.
So I've kind of been focusing entirely
on launching that new brandfor the last six months.
It's nice to take a step back
and have a cup of coffee with you
and talk a little bitmore about technology
and less about branding guidelines.
- Absolutely.
Before I throw it over to Brandon, Alex,
I wanted to start the conversation,
especially since you havethis new product coming out.
(01:47):
What are the technology trends?
What's going on in this space
that you guys launched a new line,
or especially with edge and AI
becoming more prevalent in the industry?
What are some thingsthat you see going on?
- I think we're at a nexuspoint, really, in the industry.
With AI, there's a lot of emphasis
on putting things into the cloud,
(02:08):
and there's a lot of pushback
from people that want to putthings on the edge as well.
So you've got one half going to the cloud,
the other half going to the edge,
and both of them have their own challenges
and potential setbacks.
So that's really whatwe're seeing at the moment
is customers are saying,"We want to leverage this,
"but we're not entirely surehow we can leverage this."
And it really is a sort of,
there's no perfect silver bullet.
(02:30):
So we've got to find theright path for our customers.
- It's interesting you bring that up
because obviously a lot of people
who are going to the cloud
are really looking forthings like more performance,
working with bigger datasets usually.
And on the edge, you tend to think
that you have a need for lowerpower, quick inferencing.
(02:50):
But then we see all theseGPUs and stuff coming out,
these huge, powerful GPUs.
And I wonder, number one, whatare some of the applications
that are driving thingsat the edge, you see?
And then do they really need,
do they need the performance of a GPU?
Can they get away with something else?
Like, are you seeing morelow power at the edge,
(03:11):
more performance requirements?
What is it?
- Yeah, I think poweris the key thing, right?
That's going to be themake or break for AI.
At the moment, AI is super power hungry.
It's consuming a vast amount of data.
It's really putting Bitcoin,
it's making Bitcoin look almostpower efficient right now.
With the amount of power it's consuming.
(03:33):
And I think for a lot of businesses,
people don't realize how muchpower AI applications consume
because they don't see it.
They've sort of outsourced the demand.
Like you run an AIapplication at the edge,
it's hugely power hungry.
And you have to deal with that problem,
the power and the heat at the edge.
(03:55):
If you're sending it off to a data center,
you don't see the challengesthat it brings up.
So it's easy for peopleto forget about that.
So I think reducing the power requirements
of performing these applications
is going to be a key challenge.
And that's going to make or break
whether or not AI sticksaround in this hype cycle,
depending on how youdefine AI and how it works.
And accessing those large data models,
(04:15):
being able to process things
and also absorb data andprocesses in real time.
The applications allrequire more efficient,
more energy efficient, moreheat efficient processing.
And I think that's goingto be the challenge.
- You're a marketing guy.
By the way, thanks for the rebrand.
- Sorry. (laughs)
- No, no, no.
I was going to say, I love Avnet.
(04:38):
It was sometimes confusing
which Avnet, I was referring to, right?
So I think the rebrandwas great with Tria.
So this push of a lot of these super,
what we would consider embedded,
like super big or superhigh performance processors,
based on what you said, is this marketing?
(05:00):
Are we just marketing toget, to sell more units?
Do we really need that?
And what are some usecases that you're seeing
in real world use cases?
We all hear about computervision and stuff like that,
but what's the realitylike from your customers?
- There is that speeds andfeeds elements of marketing.
So it can perform an extra amount of tops.
(05:24):
It can clock at this frequency.
It's got even more RAM.
And if I'm building my gaming PC,
then that's a sort oflike, "Oh, this is great.
I want to be able to get thisextra amount of frame rates.
I want to be able torender videos much faster."
But at the same time, you then have to,
like upgrading graphicscard at my last upgrade,
I had to get a PSU that was twice the size
(05:46):
of the previous PSU.
And you're just like, "Wow, I'mpushing a thousand watts now
to run a proper PC rig"
when it used to be like300 watts was a lot.
That's triple the amount.
For customers, that's the issue.
We had the energy crisis recently.
That brought it to the top of the agenda.
(06:06):
And now it's eased offa little bit for now,
but it's not so long before I think
it's going to come back up again.
The energy consumption, thepower is going to be critical.
So it's not so much about gettinga more powerful processor,
the most powerful processor.
It's about balancing consumption,longevity, capability,
(06:28):
specific to the application.
For customers like that, okay,there is a marketing element.
You want to buy theabsolute top of the range,
the flagship processor,
when actually you might not need that.
But sometimes you do.
And it depends on the application
for what you're going to do.
I'm the least marketing-ymarketing-y guy in that respect.
I'm kind of like, I'd rathersit down with the customer
(06:49):
and say, "Okay, tell me whatyou're actually building."
Rather than just say, "Yes,you need the top of the range.
"You need the i9 immediately."
- What are they building?
What have you seen?
- There's loads of different things
that we're working withcustomers on at the moment.
And a lot of the applications, I mean,
there's a crazy amount.
Everything from new farming applications.
(07:10):
I was reading about anopportunity mentioning no names.
I mean, there's a lot ofarticles at the moment
about more efficient farming
and artificial intelligence being used
as an alternative to thingslike dangerous forever chemicals
that are being put into the soil.
So can you train an AIrobot to move around fields
(07:31):
and identify weeds,being able to tell weeds
and pests apart from cropsand non-harmful animals,
and to be able to organize accordingly.
One of my friends worksin the farming industry
here in the UK,
and he works in a farmingmanagement industry.
So crop checking.
And he has to walk through fields,
(07:53):
taking photos of the different plants
and then educate peopleworking in the fields
to tell the difference betweenthe different varieties
of the plant and whichone to select for breeding
to build the best crop.
And you can create an AIapplication in the field
that does that for you.
You don't necessarily wantto put all of that content
into a data center.
You want to be able toprogram the robot at the edge
(08:15):
to be able to do that.
So we're seeing applicationslike that in agriculture.
And those are edge-based applications.
You don't necessarily have areliable cell data connection
all of the time.
You want to be able to do thatedge-based AI recognition.
And then at the oppositeend of that spectrum,
so you've got the massiveindustrial agriculture use case,
(08:36):
and then we've got automaticlawnmowers for people at home
and being able to map thebest path around the lawn,
but then also being able to spot hazards
and deal with hazardsaround the lawn as well.
So one is a sort of greatfuture-facing altruistic solution.
The other one is a morepractical real life solution,
but it's those practicalchallenges in the real world
(08:58):
that really put thetechnology to the test.
- Are both of those visionapplications, I'm assuming?
Like camera vision?
- Yeah, yeah.
So both of those, I mean, both customers,
one is vision, one can bemore radar sensor application,
but vision is where the jump is
(09:18):
in terms of the processing requirements.
So that live vision AI,
so being able to understandwhat it's looking at
in as quickly as possible,identify it reliably
and act on that identification
instead of having to send signals back
to a data center for crunchingand then get it back again.
It's being able to do thatin a short amount of space
and a short amount of time.
- So this is exactlywhere it's like, okay,
(09:40):
well, you got your trade off time.
It's like decision time, right?
Beause now you're saying, all right,
well, you've got vision out there
and these are probablyboth mobile, I'm assuming,
or semi-mobile, right?
And you have to send, at least
in the industrial ag use case,
you're sending that back somewhere, right?
(10:00):
So is this the place where you're like,
how many GPU execution units can I fit
into this or are you really,
with Tria now, are youtaking it case by case
and saying, look, I mean,from a cost perspective,
let's figure out form,fit and function here.
(10:21):
And it's not top of the line.
Is that the case?
- A lot of customers willhave several different tiers
of the product that they're creating.
So especially for different markets
where there's a different appetite
and also different sizesof the amount of things
that they need to crunch.
So for agriculture, you'llsee that there's the top
of the range where they wantto have mass scale farming,
(10:44):
say in America's with the giant fields
and they want to be ableto do things at speed.
They'll have the topof the range solution.
You buy something really big,it will work in the field.
It's going to cover ahuge amount of distance
in a huge amount of time for a giant farm.
So they have the money,
they have the ability to invest in that.
And then you'll want tohave a slightly slower,
slightly cheaper mid-rangeapplication as well.
(11:06):
And then you want the lower endrange as well for the market
and then let the consumer decide.
Obviously you want to sellthem the best solution,
but sometimes it's notgoing to be an option.
And it's balancing,most customers will have
various different levels of capability
and sell that to the end userbased on their application.
(11:28):
And for me, that's where theindustry is driven forward
by the actual application andwhether or not the end user
feels the need for that amount of use.
I'm always reminded of the picture
that does the rounds onthe internet of the field
and the path that leads around the corner,
(11:49):
like an L-shaped corner.
And then there's a troddenpath across the field
where people have justwalked across diagonally
and it's like designversus user experience.
And I think that likethe last cycle of AI,
there was all of thissort of exciting talk
about what was possible,but at the end of the day,
(12:09):
what was successful and wasn't successful
was defined by people actually using it
and finding it useful.
So the applications that were created,
some of them stuck around,some of them didn't.
It was the same with blockchain
when blockchain wasskyrocketing in usefulness
and the same with NFTs,Bitcoin, that kind of thing.
People actually finding ituseful as an application
(12:30):
and being able to use it every day
and decided what stuckaround and what didn't.
- The same thing seems to havehappened in the IoT space.
There were a bunch of different use cases
that were really pushedhard, like smart home stuff.
And there's a point atwhich as a consumer,
not just like a B2C consumer,but any kind of consumer
(12:51):
where you just eitherdon't need any more of that
or it's just not really practical.
It was a great proof of concept,
but it's not useful at thescale that it's being promoted.
And I think we run into thatdanger zone here with AI too,
where it's like there's alot of vision type stuff
(13:11):
that's getting pushed and it's cool.
And I know that themargins are bigger there,
but ultimately, a lot ofthe actual deployments
aren't going to be exactlywhat you see out in the media.
And it sounds like you're talking about
with the trodden pathacross the fields, right?
It's like the use cases,the demand in the market
(13:32):
is going to start driving exactlywhere this technology goes
and then how it evolves.
- Yeah, you knew that the IoT concept
had reached the top of its hype cycle
when there was IoT toasters on the market.
And okay, like we were saying before,
there's different tiers ofthe products that's available.
Some people will go for that top tier
(13:53):
and some people will just be like,
I want my toast to beslightly more toasted.
I just turn a knob on it, sameas I did back in the 1950s.
It doesn't need to beany more smart than that.
I do like, I recentlyupgraded my aging fridge
to a semi-IoT fridge thattells me if the door's open
or if the temperature'stoo high or too low.
(14:16):
And for me, like I don't need a fridge
with a screen on the front
that gives me informationabout the weather
because I've got a separatedisplay in my kitchen.
I don't need somethingwhere you knock on the door
and it shows me the products behind it.
I don't need a camera in there,
but I do like it if it warns me
if the door's been left openand it beeps on my phone.
(14:37):
And that's usually because mypartner's been loading food
into the fridge andforgot to close the door.
And then I'm in here in my room
and I'm just like, youleft the fridge door open.
Those real life applicationsare what sticks around.
So IoT is now quite a mature market
where the businesses that are investing
in that level of technology,
they put all of thetechnology into the device.
(14:59):
The consumer demand forthat sort of technology
cools off a little bit to a level
where the consumers understandwhat's beneficial to them
in their everyday life.
We've got another customerthat we're working with
that makes industrial cookers.
So for cooking consistently huge amounts
(15:20):
of the same identical foodstuffs
over and over and over again.
There's an IT model, an IoT model there
because you want to be able to control
all of the different ovens
and also manage a hundreddifferent ovens at the same time
and know if one of themis over temperature
or under temperature, that kind of thing.
There's applications therethat we're working with
where that is a requirement
(15:41):
where it might not have been 50 years ago
when cookers were being used
as an industrial scale like that.
- So unfortunately whathappens with this hype cycle
like you mentioned is thateverybody has these huge ideas,
these grandiose visions of what the future
is going to be like, wherewith IoT for example,
it was everything is going to be connected
(16:01):
and your toast is goingto be ready in the morning
and your car is going to be sitting there
waiting to drive you off to work
and it's going to beperfectly climate controlled
and by the time themarket starts to mature
and people realize it's going to cost you
a quarter of a milliondollars per consumer
to realize that vision andit's not going to happen,
everyone experiences a sort of letdown,
(16:23):
that's the trough ofdisillusionment, right?
But that doesn't mean that thetechnology is actually dead
or even unsuccessful, right?
It just means that it'sevolved in some different way
and I think what you'redescribing with AI here
and even that last industrialovens sort of example
is like, hey, there are alot of use cases out there
(16:45):
that aren't necessarily thebiggest, baddest processor,
RAM combination that youcould potentially have
but the volume's there and it exists.
- That connects up withwhat we were talking about
with power efficiency, so understanding
you get all of theinnovation, the excitement,
all the things we couldadd and then you say,
(17:07):
yeah, but I need realistically,practically to run it
with this amount of power drawin order to get what I want.
So you got to sacrifice something
in order to get something else.
Sort of like with an electric car,
you add loads of bells and whistles to it,
it gets heavier and heavier
to the point that the range drops
and then you're, well, Iwant a long range model
so I've got to increase the aerodynamics
(17:28):
which means making it looka little bit less attractive
and strip out things like power seats
in order to reduce the weight as well.
So you've got to find that middle space,
that sweet spot in thesesorts of applications.
- How does the portfolio, Tria,
like expand or develop inorder to meet that range,
(17:50):
that range of requirement?
- I think we've got a pretty good range
that goes from tiny littlelow power compute applications
all the way up to the COM-HPCs
with the server gradeIntel processors in them.
So, and like the COM-HPCswith the Intel processors
(18:10):
are designed foredge-based image processing
and AI applications, butthey're larger as well.
So you have to have a balance between size
and power consumption andwhat they're capable of.
So a lot of the larger,the COM-HPC modules
(18:31):
are this sort of size, they'resort of motherboard sized
which means that you've got to put them
inside a dedicated case.
You couldn't just embedthem directly into a product
unless it was a really big product.
So for things like edge securityor public transportation,
so AI applications andpublic transportation
is another thing that we'reworking on at the moment.
Being able to take data froma huge number of sensors
(18:54):
from a train or othervehicle or train station,
analyze them all, reactto them in real time.
That pretty much requiresan on location server
because sometimes you can'trely on the data network
being reliable.
And that means that we're using those
for those sorts of applicationsin standalone servers.
But a lot of the requirements,
(19:14):
we've got ones for industrial automation.
So again, we're workingwith Intel on cobotics
with one of our customers,building real time image sensors
into a cobotics, a cooperativerobotics environment.
So a robot can operate in thesame space as a human safely.
So if the human moves into that space,
(19:36):
the robot arm stopsmoving, can move around.
If the human picks something up,
the robot knows where it isand can take it off them again.
We were demonstratingan early example of that
at Embedded World in Nuremberg this year.
That was built around a combination
of the Intel based ComExpressmodules that we have
and the Intel based, actually no,
(19:56):
it's Intel based SMARC modules.
And then our Intel based COM-HPC modules
for the image processing.
And those two thingscommunicating with each other.
So getting the signalsfrom the cameras analyzed
and then communicating withthe robot in real time as well.
So there is that sort of,
how useful is the environmentthat you're creating,
the application that you're creating there
(20:17):
versus the amount of power,
the amount of processing that you need,
the amount of space you needin that environment as well.
For some customers, that's a pinnacle.
So it's giving them the option to say,
okay, well, I need cobotics,
I need to have a reliable environment
and therefore I need thatextra processing power
and the associated costs thatcome with setting that up
(20:37):
and developing and installing it.
Whereas other manufacturers,
they might want to just havean enclosed robotic space,
no cobotics required.
You got to, like you were saying before,
you have to create thepotential for innovation.
So you have to inspire the customers
with that new technology,that new possibility,
and then let the customer
(20:58):
then build that application around it.
And if it works for them,
then that creates the foothold
for that technology to develop further.
And for sometimes you'llcreate that new technology
like a lot of the AI applications
that we're seeing at the moment
where the customer, theuser can't really find
that sort of really, that killer app point
(21:19):
that becomes a foothold forthe technologies to develop.
- Tria has almost used the bad A word now,
the old A word.
(laughs)
Within the Tria Portfolio,
obviously it's pretty expansive, right?
I mean, there's lots of options.
What is the Intel portfolio look like?
I mean, are you offering Atom, Core, Xeon,
(21:39):
you know, the sort of the gamut
or what does that looklike in terms of scale?
- Yeah, pretty much the full gamut.
I think within the mobile processor space,
like I said, up to the COM-HPC level,
we can put server gradeprocessors onto those.
But at that point you may aswell have an actual server.
(22:01):
So it depends on the size, the shape
that you need to put it into.
So yeah, we typically offer the Atom
and the Core series and theXeon series at the server end.
We have those, it's really cool to see
what the product team does,
putting things into such a small space.
I've been working with motherboards
(22:22):
and processes for motherboards
for years and years and years.
So to see that sort ofcomputing application
in such a small packagewith heat management,
thermal management is a fine art.
And watching the team developthose sorts of applications
in the environment that theproduct's going to be used in
is a fascinating challenge.
(22:43):
So being able to deploylike an Intel processor
and its capabilities andthe new AI based processes
we're working on as well,
to bake those into a small product
to be able to use at theedge is pretty exciting.
- Well, cool.
I mean, it's really exciting to see
more of the AI in action thanAI in advertisement, right?
(23:05):
So really looking forward toseeing how this continues.
- Brandon I actually wanted to ask you
because you covered embeddedworld for us this year,
which feels like it was lastyear at this point already,
but there was a lot of nextgeneration edge processors
that came out that Intel launched
Intel new core processorscalled Ultra Intel® Arc™ GPU.
(23:27):
So I'm curious, what have you been seeing
around the industry, especiallyas we've talked about
all these use cases and constraints
of how the latest processorsand technology advancements
are helping some of thepartners in this space?
- There's obviously like the software side
and the silicon side,on the software side,
you've got DevCloud and OpenVINO™
has got a really good foothold,
(23:48):
really helping streamline and accelerate
the development of models.
And there's even Intel® Geti™
which is even furtherback on the training side
and just making it easier there.
On the Silicon side, man, the core Ultras,
like the AI PCs, I think thatthey're a really nice fit
(24:09):
in this sort of spectrumthat Alex is talking about
because they enable youto scale up and scale down
even with inside the same skew, right?
Because you've got alot of different compute
that's available to you.
These heterogeneousprocessors where you can say,
look, I want a performance core,
I want an efficiency corefrom a CPU standpoint,
(24:31):
but then also, they've gotgraphics execution units built,
integrated GPUs where youcan do acceleration there.
And then you bring in neural accelerators.
So, you can get this sort of ability
to move your application inone direction or the other
based on what is availableon the SoC or chip set.
(24:53):
And that just givesyou so much flexibility
because at that point toAlex's point about efficiency
and power consumption,you're using the right core
for the right workload, right?
And that's really ultimatelywhat it's all about
because that allows something
that would traditionallyhave been a smaller
or less expensive processorto accomplish more.
(25:17):
And really that's kind ofthe name of the game here.
- That's a really good point, Brandon.
I was at Intel's AI event recently.
They had that globalevent where they showcased
all of their latest AI technologies.
The applications there tolook at some of the partners
that we're showcasing were fascinating
for how you can take AI to accelerate
(25:39):
an application at the edge.
There were things likesupermarket checkout applications
which were automaticcheckouts that recognize
what it is you're holdingand queue management,
automating supermarket management as well.
But it was really coolto see the applications
that Intel was developing at the Olympics.
(26:00):
So, the athlete applicationsthat they developed there,
that's a really great wayof taking the technology
and showing a real life usecase to capture the imagination
of potential developers of the technology.
And the case study video that they showed
of the technology being used in Africa
(26:21):
to sort of scout a hugenumber of potential athletes
and then find potential future Olympians
based on image processingusing that platform.
That was a really cool, thatreally captured my imagination.
It really stuck with me.
But being able to take thatAI processing to the edge
(26:41):
and in a laptop as well,
it goes back to what we weresaying at the beginning,
taking that high power today, it's hugely,
they're large units,they're very powerful,
compressing it, making it smaller,
making it more energy efficient,
being able to put an AIapplication into a laptop,
a laptop-sized device thatcan be used in the field
(27:02):
is really exciting.
I think it was Dell that was up on stage
that was showing the laptops
that they're going to be releasing
with built-in AI applications.
So it's an AI device insteadof a computing device
and really leaning into that collaborative
AI application environment.
You've got a greatexample from the Olympics
(27:23):
that Intel's done, but it's a blank slate.
I'm really excited tosee what developers do
with that amount of AI processingtechnology at the edge,
instead of having todepend on sending stuff
back to a huge data center and back again.
And I think that's going tobe a turning point really
for the future for AI at the edge.
(27:43):
- Honestly, I think a lot'sgoing to be about sustainability
and something I forgot to bring up was,
man, have you ever put afarmer's market piece of produce
next to a supermarket piece of produce?
It's weird.
- I don't know where you'regoing with that, Brandon.
- Well, you were talking aboutnot using chemicals, right?
(28:04):
Not having to use as many chemicals.
And when you put the farmer'smarket versus the supermarket
like something is not right here.
But sustainability in the future,
I think is really important
and I think all the thingsthat you've been talking about
and we've discussed todaywill help us on that path.
- Yeah, for sure.
- It's amazing, allthe different use cases
(28:25):
and everywhere you can gowith these AI applications.
I can't wait to see where else we go,
especially with partners like Avnet.
So it's been a great conversation, guys.
Thank you for joining.
Before we go, Alex, I justwant to throw it back to you
one last time, if there'sany final thoughts
or key takeaways you wantto leave with us today.
- Like I said at the beginning
and like we were kind of leadingback into at the end there,
(28:45):
I think that AI is at anexus point at the moment
and I think edge computingis a nexus point as well.
So that's advancement inedge-based AI applications.
So being able to takeit away from the cloud
and onto the device,that's the nexus point.
If you're watching this,find those applications
and tell us about them if you've got them.
I think it's a reallyexciting time to be working
(29:07):
in computing on a small formfactor with AI in this space.
- Yeah, and I invite all of our listeners
to visit the Avnet website,
check out their new product line,
see how they can help youtake some of your AI efforts
and initiatives off the ground.
So thank you both again, Brandon,
it's always great connecting with you.
You've always been ourembedded systems expert
and thanks to our listenersand thanks Alex from Avnet.
(29:31):
Until next time, this hasbeen "insight.tech Talk."
(upbeat music)
Popular Podcasts
United States of Kennedy
United States of Kennedy is a podcast about our cultural fascination with the Kennedy dynasty. Every week, hosts Lyra Smith and George Civeris go into one aspect of the Kennedy story.
Stuff You Should Know
If you've ever wanted to know about champagne, satanism, the Stonewall Uprising, chaos theory, LSD, El Nino, true crime and Rosa Parks, then look no further. Josh and Chuck have you covered.
Dateline NBC
Current and classic episodes, featuring compelling true-crime mysteries, powerful documentaries and in-depth investigations. Follow now to get the latest episodes of Dateline NBC completely free, or subscribe to Dateline Premium for ad-free listening and exclusive bonus content: DatelinePremium.com