July 22, 2025 • 23 mins
Unlocking Zephyr with NXP: Application Code Hub & Facial Detection Demo
Join the EdgeVerse Techcast as Kyle Dando and Bridgette Stone delve into the world of NXP's Application Code Hub. Special guest Derek Snell, an embedded development expert, discusses the Zephyr RTOS and its integration into NXP solutions. Learn about the innovative Facial Detection demo, which showcases the portability and NPU performance of the MCX N947 microcontroller. Discover how the centralized Application Code Hub simplifies the evaluation and development process across different hardware platforms. Ideal for software engineers eager to explore Zephyr and streamline their development workflows with NXP's powerful tools and demos.
Zephyr App Code Hub Examples:
- https://mcuxpresso.nxp.com/appcodehub?search=dm-facialdetect-zephyr-demo
- https://mcuxpresso.nxp.com/appcodehub?search=dm-smartwatch-zephyr-demo
- https://mcuxpresso.nxp.com/appcodehub?appFormat=zephyr
- http://www.nxp.com/zephyr
00:00 Introduction to EdgeVerse Techcast
01:21 Meet Derek Snell: Zephyr Expert
02:47 Zephyr Demos in Application Code Hub
07:18 Facial Detection Demo Overview
12:09 Technical Insights and Comparisons
17:34 Upcoming Demos and Zephyr Advocacy
22:13 Conclusion and Future Episodes
Mark as Played
Transcript
Episode Transcript
Available transcripts are automatically generated. Complete accuracy is not guaranteed.
(00:11):
Hey everyone, welcome backto the EdgeVerse Techcast.
This is your favorite spot for all thingsNXP software, tools, and enablement.
Especially for NXP's processorsand microcontrollers.
I am one of your co-hosts, KyleDando, and today we're jumping
back into the Application Code Hub.
(00:31):
That's right.
That's a treasure chest of ready torun projects that help you develop and
evaluate applications for the real world.
And they're all using the NXPsolutions that are available for you.
And joining us today is someone wholives and breathes embedded development.
He's a proud Rose-Hulman grad thatmakes him a true fighting engineer.
(00:55):
Oh, but don't worry about that.
It's not the punch me inthe face kind of engineer.
It's the more build somethingthat changes the world type.
All right, Kyle, we get it.
You've got a bit of a man crush.
This is Bridgette Stone, your otherco-host here to keep things moving.
And yes, today's guest is DerekSnell, engineering whiz, and our
(01:17):
go-to guy for Zephyr examplesin the Application Code Hub.
Derek, welcome to the show.
Why don't you tell usa bit about yourself?
Wow, that was quite the intro there.
I'm not used to that kind of fanfare,but thank you for having me on your show.
I did go to Rose-Hulman.
Was a, I guess, a fighting engineer.
It's been a long time, feels like, butyeah, I thought that nickname was always
(01:41):
a little violent and kind of ironicconsidering it's a school full of nerds.
But Kyle, I like how you lookat it as changing the world.
That's a good outlook there.
Thank you.
So, after I was officially a fightingengineer, I eventually made my way to NXP.
I've been here a long time.
I spent most of my time here at thiscompany as a Field Application Engineer,
(02:02):
working closely with our customers.
And then a few years ago I transferredinto our microcontroller product line.
They called me a Systemsand Application Engineer.
I spent most of that time working withZephyr and I'm very involved in supporting
Zephyr, so I support our customers.
I support our internal teams.
(02:23):
I'm working a lot with our Zephyr teamsto proliferate Zephyr more across our
devices and expand the functionalityand driver support that we have.
And I work with Kyle and thetools teams to make it easier
to use Zephyr and learn it.
So for me, it's allabout Zephyr these days.
Well, we're so excitedto have you here, Derek.
(02:44):
Thank you.
And I'm gonna juststart it off at the top.
How are you involved in getting Zephyrexamples into our Application Code Hub?
Right.
We just pushed a couple demos I've beeninvolved with into Application Code Hub.
We have a couple demos thatwe have created for Zephyr.
We wanted to showcase some nice featuresof using Zephyr as well as our hardware
(03:05):
and some of the value add that we bring.
We created these demos.
We released them publicly.
The one we're gonna talk about todayhas been out for over a year now.
But our problem is people finding them.
We try to promote 'em and we putlinks out there, but i'm constantly
talking about these demos withpeople inside NXP and outside, and
(03:25):
they're totally not aware they exist.
Then Kyle and the team camealong with Application Code Hub,
and this has been a great tool.
I'm really glad we have it.
Thank you for bringing this out becausehaving one place to go look for this
stuff, it's a game changer, in my opinion.
And so we've released some ofthese effort demos up there.
They're easy to find now and the tooltakes you right to 'em that it's built
(03:49):
into VS Code, we can talk about today.
So it's just that much easierfor us to deliver these demos and
make sure that they're found andpeople can take advantage of them.
Yep.
It's not just showing about howit works, but giving developers
a shortcut to see it in action.
Exactly.
So you started to talk about it alittle bit, Derek, which was the
(04:11):
integration of the Application Code Hub.
You mentioned sharing links ordriving people to 12 different places.
Really, the Application CodeHub is trying to have a central
place where everybody can go.
Why don't you explain to the listenersthe different ways that they can
use the Application Code Hub?
Whether it's in the IDEor maybe the command line.
(04:32):
Talk about how you see people oncethey find your Zephyr applications.
How do you see them getting andworking with those projects?
Right.
Well, one of the nice features ofApplication Code Hub is that it is
integrated in our tools like VS Code.
And so right within that IDE youcan browse Application Code Hub.
(04:54):
You can filter and you can findthese apps that you're interested in.
You can read about 'em.
And if you wanna play with 'emand test them out, you just click
a button and it helps you importit right into the tool and makes
it that much easier to play with.
I think in the Zephyr space, that'sreally helpful because getting
everything set up and learning Zephyr.
Zephyr has a steep learning curve.
(05:16):
Anything we can do to make that easieris helpful for people learning that
stuff and getting it to play with asample without having to invest a lot
of time learning the mechanics of it.
So that's enabled.
We have a nice MCUXpresso extensionfor VS Code, which makes it easy to
set up your Zephyr environment, importthese sample applications and find the
(05:38):
Application Code Hub demos that apply toyou and bring 'em in and test them out.
Flash our boards and debug'em, so that's great.
But in the Zephyr world inparticular, a lot of our developers
we work with, they do command line.
Zephyr's been doingthat from the beginning.
Zephyr uses this tool "west" andbuild tools to have a consistent build
(06:00):
environment and flow across differentplatforms so you're not stuck using
different vendors tools all the time.
A lot of Zephyr users preferthat method anyway, so that's
fully supported here too.
So when you go to Application Code Huband you find a sample, the READMEs also
tell you how to do it from command line.
(06:20):
If you're familiar with Zephyr, it's thesame method of using like "west init"
commands to initialize your workspace and"west build" commands to build it all.
So, it fits right in with thestandard Zephyr development.
Yeah, and I think for the listeners,the Application Code Hub, Bridgette and
I have talked about several episodes.
It's basically a really nice front end.
(06:41):
A user experience like Amazon.com,where you can go in and find things.
But the shelves where theapplications are, it's GitHub.
So like you said, Derek, if you'rea knowledgeable Zephyr developer,
you're probably working withinthat command line environment.
You use Git quite a bit justbecause of Zephyr's distribution.
And so they can bypass thefancy shelves and go straight to
(07:04):
GitHub and do those command lineinteractions with your projects.
It makes it available for both audiences: those that want it hand delivered on (07:08):
undefined
a silver platter, and for those thatcan actually cook it themselves and
want to get it directly from GitHub.
Okay.
So time for the star ofthe show Facial Detection.
So Derek, what does this project doand which NXP platforms are supported?
Right, so one of the new demos wehave on Application Code Hub based
(07:31):
on Zephyr is a facial detection demo.
And as I mentioned this is,it's been around a while.
We created it last year for someevents and trainings that we did.
We wanted to highlight one of thekey features of Zephyr that attracts
users, which is the portability andbeing able to run the same application
across different hardware platforms.
(07:52):
So this demo focuses ontwo hardware platforms.
One is the MCX N947, which was our firstMCX microcontroller that we launched.
And we wanted to showcase thatpart 'cause it was new and it has
a neural processing unit on it toaccelerate machine learning algorithms.
So we wanted to take advantage of that.
(08:13):
We settled on doing this FacialDetection demo porting it to Zephyr,
getting it working in that environment,using the Zephyr drivers and the NPU.
Then we ported it to a differentplatform, which is the i.MX RT1060.
A very different architecture.
It has a lot of differenthardware features.
(08:36):
But it ends up being the same application,same source code, running on the
two different platforms, even thoughunder the hood they're very different.
So that was the intent of this demo.
And this version is in the NXPdownstream, Zephyr repo, not upstream.
Correct?
And why is that?
What sets it apart from theupstream facial recognition project?
(08:58):
Right.
So the application itself doessit in a separate application
repo that we have on GitHub.
It's not in the upstream.
But it uses some upstream repos.
That's one of the beauties of west thatZephyr leverages is they can pull in
Zephyr and other modules, other repos, andyou can specify the location where those
(09:22):
repos are pulled from, whether they'reupstream or downstream, or someone's
fork, and you can specify the revision.
West makes that easy to pick andchoose the pieces that we want
to bring into the workspace.
But talking about downstream.
This demo leverages TensorFlow LiteMicro for the machine learning.
Zephyr also integrateswith TensorFlow Lite.
(09:44):
By integrating it, it makes it reallyeasy to use in a Zephyr application.
It uses west, it pulls it inas a module and it's got some
examples to make it easy to use.
But we ended up using the downstreamrepo that NXP has for that.
The main reason is becausewe take advantage of the NPU
on the MCX microcontroller.
(10:05):
So we have a fork of TensorFlowLite Micro that accelerates the
machine learning on that NPU.
When we created this demo, it was veryeasy to just update the west manifest
and select that we wanted this downstreamport from NXP, and it gave us instantly
this ability to accelerate on the NPU.
Excellent.
(10:25):
That explains it to me.
'cause I was wondering the differentiationto the customer when they go to upstream.
NXP fully supports the upstream, butas you just mentioned, when there are
some NXP specific IP blocks or supportthat is not currently in the upstream
repo, like taking advantage of ourNeural Processing Unit, it definitely
(10:46):
makes the downstream nice to have.
You can quickly get thesedemonstrations out as you work to
integrate it with the upstream.
You don't have to wait.
I know we use the downstream Derek forsome new product introductions that
haven't made it to the Zephyr stream yet.
We release them in our downstream andas soon as they're readily available
on the upstream we get them into there.
(11:07):
Is that also correct?
Yeah, that's exactly right.
At NXP we, we want to upstream and workwith the upstream as much as we can.
We have, with our Zephyr development,we have an upstream first mentality.
And nearly everythingwe contribute upstream.
That's where it goes.
But we do have a downstream and we havethese various forks around and it's
(11:28):
like you said it's mainly for stuffthat either we can't upstream right
now, maybe 'cause we don't have fullsupport out there, or in the Zephyr
case they have strict licensing rules.
So sometimes when we enable some ofour IP and hardware that we have with
the licenses we have, we're not able tostick 'em in as Zephyr's upstream repos.
(11:49):
So now we have the downstream.
We have full control over that, wherewe can push stuff in there to make
time commitments that we make tocustomers, and then eventually get
it into the upstream when we can.
Perfect.
Thanks for expanding on that a little bit.
I think our listeners will appreciate it.
All right, so Derek I have seenso many facial detection demos.
I'm sure our listeners, those are, yeah,they're evaluating examples and they say,
(12:13):
oh, yet another facial detection demo.
Maybe you can spend a fewminutes to discuss what are you
trying to technically highlight?
You mentioned the NPU acceleration, someof the other software that we provide, the
portability in an i.MX RT versus an MCX.
But can you differentiate this facialdetection demo and the technical
(12:37):
advantages that you think thecustomers should take a look at?
Yeah, and that's a fair question.
Going back to the history of thisdemo, like I said, we were putting
this together at the time that wewere launching the MCX N device,
which was, it had the new NPU on it.
And that was a big featurewe wanted to promote.
To your point, there was alreadya facial detect demo that we had
(12:59):
done for that device to showcasethat hardware and that silicon.
That was a bare metal demo, meaningit, it did not use any RTOS.
So we thought it would be a great ideato take that demo to port it into Zephyr.
So for one thing, it shows the integrationthat we have for machine learning and
Zephyr and being able to use the NPU.
(13:19):
But then we also wantedto show this portability.
We thought it would be a greatidea to contrast the performance
advantage that the NPU gives you.
So we compared it to the RT1060.
which, it is a muchhigher performance device.
You know, the MCX microcontrolleris 150 MHz, M33 core.
(13:40):
The RT1060 runs at 600MHz and it's an M7 core.
But when you look at this demo and it'san apples-to-apples comparison 'cause it's
literally the same code running on both.
The inferencing time on the MCXdevice is 22 milliseconds and it's
five times longer on the RT1060.
(14:01):
Wow.
I think that tells a great story.
I mean, you got a part that's runningfour times faster, but it takes five
times longer to do the same inferencing.
So it does show the advantage of the NPU.
It also shows how we can take thesame app and scale it around because
there are advantages in the RT family.
It does have more integration.
It's got better graphicsand camera interface.
It's got a lot more performancethat I can offer beyond the NPU.
(14:25):
So it makes sense in manyapplications to use that device.
You get a lot of our customersand users out there who, they
don't just have one product.
They make a family of products,they have a portfolio, they have
like a Good, Better, Best modelthat they sell, and they wanna reuse
as much of that code as they can.
And so they're frequently forcedto have to use different hardware
(14:47):
platforms and they don't wanna haveto reinvent the wheel all the time.
So this is a demo showing wherewe have a real world application
on these different platforms.
And beyond the NPU, it also shows there'sa lot of differences under the hood.
This demo uses a camera to look at yourface, and it's got a display where it
shows what the camera sees, and it'll drawrectangles around the faces it detects.
(15:10):
The RT1060 has a trueCSI camera interface.
It's that a high end displayinterface, but it has a lot
of features built into it.
On the MCX side it doesn't have thosesame interfaces, so it, it doesn't
have a camera interface that doesn'thave a true display interface.
So we ended up using, a SmartDMA, it'scalled, it's a special peripheral we
(15:32):
have on that family that we can configureand that became our camera interface.
When we did this integration in Zephyr.
Zephyr has a, a common video driver that'sable to hook up to cameras and it provides
this, you know, same interface, same APIsthat the application could call regardless
of the hardware they're running on.
And in the RT family, we already hadthat video driver available, with some
(15:57):
examples showing it with our camera.
But when we ported this demo to MCX.
We didn't have that driveravailable at the time.
So we ported Zephyr's video driver tothis SmartDMA, but in this application,
that device is limited on RAM.
'cause you got a, a lotgoing on in the app.
(16:18):
You got the machine learning model,you got the display that's got
frame buffers, and then you gotthis camera that you're bringing in.
So what the bare metal applicationhad done was it brought in a partial
frame from the camera into a smallerbuffer and processed it instead
of bringing in the full frame.
That was a way to save RAM.
(16:39):
So when we ported the driver to Zephyr.
The Zephyr APIs didn't havethat capability, at that time.
They only brought in a full frame Buffer.
So NXP contributed this new feature.
This is frequently what we do aswe're enabling our hardware and
adding demos into the Zephyr world,is we also contribute to Zephyr that
(17:01):
helps with everyone using Zephyr.
So in this case we added this abilityto do these partial frame to the
driver, and we got that upstreamed.
Then that enabled us to usethe same Zephyr video drivers
on these different platforms.
Even though under the hood they're manageda lot differently at the hardware level.
So, very interesting.
And I know you mentioned that theFacial Detection is one of the
(17:22):
first applications shared on ACH.
What other applications are youseeing being added in the future?
Is there something you canshare with us to look forward
to sometime in the near future?
Yeah, there's a very cool one coming.
I, I recently heard your episode withEduardo Viramontes, who also referenced
(17:42):
that there's a Doom demo, the videogame Doom, that runs on Zephyr.
It runs on the same MCX N947FRDM board that we have.
That demo's been out as well.
It's very cool.
It's fun to see and it'scoming to Application Code
Hub, so that's pretty exciting.
I think that's gonna be funwanting to get up there.
All right.
Awesome.
I'm looking forward to that.
(18:03):
We'll be sure to drop the links inthe episode notes so our listeners
can explore the demos for themselves.
Sure, sure.
And this has been great Derek,but we've had a few episodes on
Zephyr and I think we would miss anopportunity if we didn't ask you.
You mentioned it's pretty much 110% ofyour job to advocate and support Zephyr.
(18:25):
So could you just in a couplesentences, what's your elevator pitch?
When you talk to somebody and theysay, "Yeah, I've heard about Zephyr.
I don't get it." How do you get peopleexcited about what you're excited about?
Right.
We have a lot of customers atNXP that have moved to Zephyr and
(18:46):
they've made this decision acrossthe board that they wanna use Zephyr.
They're very engaged with us,and there's definitely some
common themes for why they do it.
And NXP, we have our own reasonsas well, but they're very similar.
I've talked about the portability.
I'd say that's probablythe biggest reason.
At NXP, we love it because wedo have this very broad, massive
(19:06):
portfolio that is very diverse.
Leveraging something like Zephyr thathas this very well defined Hardware
Abstraction Layer makes it easier forus to focus on applications and demo
stuff like this that gets our customersthat much closer to production.
And then reuse it across our portfolio.
But our customers need that same typeof portability and scalability as well.
(19:30):
They don't want to spend all theirtime fussing with drivers and stitching
middleware together and everything.
Zephyr does a lot of that for you.
I think the other key thing is justthe big ecosystem that is Zephyr.
I also heard your episode youhad recently with Eli Hughes.
He does a great job pitchingZephyr as an ecosystem.
(19:51):
Sometimes I talk to people who arejust learning about Zephyr and they had
this perception that Zephyr's just thekernel, like some other RTOSes out there.
They think, "I don't need anotherRTOS", but Zephyr's way more than that.
It has this hardware abstraction layer.
It's got a lot of middleware,either directly integrated as
part of the Zephyr project, or theintegrated as third party modules.
(20:16):
So things like USB Stack, ethernetStack, Bluetooth, and Wi-Fi.
Those things are native stacks in Zephyr.
But what I like about Zephyr is thatif there's already valuable open
source software out there that'savailable for middleware for other
features outside what Zephyr has.
(20:37):
If it's well supported and hasgood community engagement, then
Zephyr will just integrate that ininstead of reinventing the wheel.
It makes it very easy to leveragethose packages within Zephyr and
they'll add examples and stuff too.
So out of box you can have examplesfor these different packages.
You can get 'em running onthe hardware you select.
(20:58):
So like we mentioned TensorFlowLite Micro, that's one of them.
LVGL for graphics is anotherone that gets integrated.
There's like Mbed-TLS andPSA crypto for security.
And the list goes on and on.
There's Mcuboot as a boot loader, butthere's something like 70 other repos that
get pulled in if you just go out there andinitialize the upstream Zephyr workspace.
(21:21):
The other aspect of that ecosystemis also the build environment.
Because Zephyr also includes whatthey call the Zephyr SDK, which is
the collection of build tools needed.
It provides this common interface throughWest that gives you the ability to
have the same development environment,the same commands that you can use
across these different platforms.
It's a very cohesive, massive ecosystemthat gives you all these building
(21:45):
blocks and makes it easy to getstarted jump into your application.
Well, Derek, thank youfor the elevator pitch.
We loved having that at the end, anda big thanks for walking us through
how Zephyr's come into life in theApplication Code Hub from facial detection
on the MCX N to what's coming next.
I'm really lookingforward to the Doom demo.
It's clear ACH is making it easier thanever for developers to evaluate real
(22:09):
world applications with NXP hardware.
Yeah.
Thanks for having me.
Well, again, for our audience, whetheryou prefer command line or a wizard
based tool, the Application CodeHub, it's delivering a streamlined
experience for our customers.
Plus it's got all that soliddocumentation Derek talked about,
and it'll get you started with Zephyrand some of these cool applications.
(22:30):
And as Derek mentioned, there's afew new things coming down the pipe.
Check out the links in theepisode notes to explore the
demo and other Zephyr projects.
And as always, like follow andsubscribe to stay up-to-date with the
latest from the EdgeVerse Techcast.
And to our listeners (22:43):
always stay curious, always keep innovating,
and Bridgette and I'll see youon the next EdgeVerse Techcast.
Hey everyone, welcome backto the EdgeVerse Techcast.
This is your favorite spot for all thingsNXP software, tools, and enablement.
Especially for NXP's processorsand microcontrollers.
I am one of your co-hosts, KyleDando, and today we're jumping
back into the Application Code Hub.
(00:31):
That's right.
That's a treasure chest of ready torun projects that help you develop and
evaluate applications for the real world.
And they're all using the NXPsolutions that are available for you.
And joining us today is someone wholives and breathes embedded development.
He's a proud Rose-Hulman grad thatmakes him a true fighting engineer.
(00:55):
Oh, but don't worry about that.
It's not the punch me inthe face kind of engineer.
It's the more build somethingthat changes the world type.
All right, Kyle, we get it.
You've got a bit of a man crush.
This is Bridgette Stone, your otherco-host here to keep things moving.
And yes, today's guest is DerekSnell, engineering whiz, and our
(01:17):
go-to guy for Zephyr examplesin the Application Code Hub.
Derek, welcome to the show.
Why don't you tell usa bit about yourself?
Wow, that was quite the intro there.
I'm not used to that kind of fanfare,but thank you for having me on your show.
I did go to Rose-Hulman.
Was a, I guess, a fighting engineer.
It's been a long time, feels like, butyeah, I thought that nickname was always
(01:41):
a little violent and kind of ironicconsidering it's a school full of nerds.
But Kyle, I like how you lookat it as changing the world.
That's a good outlook there.
Thank you.
So, after I was officially a fightingengineer, I eventually made my way to NXP.
I've been here a long time.
I spent most of my time here at thiscompany as a Field Application Engineer,
(02:02):
working closely with our customers.
And then a few years ago I transferredinto our microcontroller product line.
They called me a Systemsand Application Engineer.
I spent most of that time working withZephyr and I'm very involved in supporting
Zephyr, so I support our customers.
I support our internal teams.
(02:23):
I'm working a lot with our Zephyr teamsto proliferate Zephyr more across our
devices and expand the functionalityand driver support that we have.
And I work with Kyle and thetools teams to make it easier
to use Zephyr and learn it.
So for me, it's allabout Zephyr these days.
Well, we're so excitedto have you here, Derek.
(02:44):
Thank you.
And I'm gonna juststart it off at the top.
How are you involved in getting Zephyrexamples into our Application Code Hub?
Right.
We just pushed a couple demos I've beeninvolved with into Application Code Hub.
We have a couple demos thatwe have created for Zephyr.
We wanted to showcase some nice featuresof using Zephyr as well as our hardware
(03:05):
and some of the value add that we bring.
We created these demos.
We released them publicly.
The one we're gonna talk about todayhas been out for over a year now.
But our problem is people finding them.
We try to promote 'em and we putlinks out there, but i'm constantly
talking about these demos withpeople inside NXP and outside, and
(03:25):
they're totally not aware they exist.
Then Kyle and the team camealong with Application Code Hub,
and this has been a great tool.
I'm really glad we have it.
Thank you for bringing this out becausehaving one place to go look for this
stuff, it's a game changer, in my opinion.
And so we've released some ofthese effort demos up there.
They're easy to find now and the tooltakes you right to 'em that it's built
(03:49):
into VS Code, we can talk about today.
So it's just that much easierfor us to deliver these demos and
make sure that they're found andpeople can take advantage of them.
Yep.
It's not just showing about howit works, but giving developers
a shortcut to see it in action.
Exactly.
So you started to talk about it alittle bit, Derek, which was the
(04:11):
integration of the Application Code Hub.
You mentioned sharing links ordriving people to 12 different places.
Really, the Application CodeHub is trying to have a central
place where everybody can go.
Why don't you explain to the listenersthe different ways that they can
use the Application Code Hub?
Whether it's in the IDEor maybe the command line.
(04:32):
Talk about how you see people oncethey find your Zephyr applications.
How do you see them getting andworking with those projects?
Right.
Well, one of the nice features ofApplication Code Hub is that it is
integrated in our tools like VS Code.
And so right within that IDE youcan browse Application Code Hub.
(04:54):
You can filter and you can findthese apps that you're interested in.
You can read about 'em.
And if you wanna play with 'emand test them out, you just click
a button and it helps you importit right into the tool and makes
it that much easier to play with.
I think in the Zephyr space, that'sreally helpful because getting
everything set up and learning Zephyr.
Zephyr has a steep learning curve.
(05:16):
Anything we can do to make that easieris helpful for people learning that
stuff and getting it to play with asample without having to invest a lot
of time learning the mechanics of it.
So that's enabled.
We have a nice MCUXpresso extensionfor VS Code, which makes it easy to
set up your Zephyr environment, importthese sample applications and find the
(05:38):
Application Code Hub demos that apply toyou and bring 'em in and test them out.
Flash our boards and debug'em, so that's great.
But in the Zephyr world inparticular, a lot of our developers
we work with, they do command line.
Zephyr's been doingthat from the beginning.
Zephyr uses this tool "west" andbuild tools to have a consistent build
(06:00):
environment and flow across differentplatforms so you're not stuck using
different vendors tools all the time.
A lot of Zephyr users preferthat method anyway, so that's
fully supported here too.
So when you go to Application Code Huband you find a sample, the READMEs also
tell you how to do it from command line.
(06:20):
If you're familiar with Zephyr, it's thesame method of using like "west init"
commands to initialize your workspace and"west build" commands to build it all.
So, it fits right in with thestandard Zephyr development.
Yeah, and I think for the listeners,the Application Code Hub, Bridgette and
I have talked about several episodes.
It's basically a really nice front end.
(06:41):
A user experience like Amazon.com,where you can go in and find things.
But the shelves where theapplications are, it's GitHub.
So like you said, Derek, if you'rea knowledgeable Zephyr developer,
you're probably working withinthat command line environment.
You use Git quite a bit justbecause of Zephyr's distribution.
And so they can bypass thefancy shelves and go straight to
(07:04):
GitHub and do those command lineinteractions with your projects.
It makes it available for both audiences: those that want it hand delivered on (07:08):
undefined
a silver platter, and for those thatcan actually cook it themselves and
want to get it directly from GitHub.
Okay.
So time for the star ofthe show Facial Detection.
So Derek, what does this project doand which NXP platforms are supported?
Right, so one of the new demos wehave on Application Code Hub based
(07:31):
on Zephyr is a facial detection demo.
And as I mentioned this is,it's been around a while.
We created it last year for someevents and trainings that we did.
We wanted to highlight one of thekey features of Zephyr that attracts
users, which is the portability andbeing able to run the same application
across different hardware platforms.
(07:52):
So this demo focuses ontwo hardware platforms.
One is the MCX N947, which was our firstMCX microcontroller that we launched.
And we wanted to showcase thatpart 'cause it was new and it has
a neural processing unit on it toaccelerate machine learning algorithms.
So we wanted to take advantage of that.
(08:13):
We settled on doing this FacialDetection demo porting it to Zephyr,
getting it working in that environment,using the Zephyr drivers and the NPU.
Then we ported it to a differentplatform, which is the i.MX RT1060.
A very different architecture.
It has a lot of differenthardware features.
(08:36):
But it ends up being the same application,same source code, running on the
two different platforms, even thoughunder the hood they're very different.
So that was the intent of this demo.
And this version is in the NXPdownstream, Zephyr repo, not upstream.
Correct?
And why is that?
What sets it apart from theupstream facial recognition project?
(08:58):
Right.
So the application itself doessit in a separate application
repo that we have on GitHub.
It's not in the upstream.
But it uses some upstream repos.
That's one of the beauties of west thatZephyr leverages is they can pull in
Zephyr and other modules, other repos, andyou can specify the location where those
(09:22):
repos are pulled from, whether they'reupstream or downstream, or someone's
fork, and you can specify the revision.
West makes that easy to pick andchoose the pieces that we want
to bring into the workspace.
But talking about downstream.
This demo leverages TensorFlow LiteMicro for the machine learning.
Zephyr also integrateswith TensorFlow Lite.
(09:44):
By integrating it, it makes it reallyeasy to use in a Zephyr application.
It uses west, it pulls it inas a module and it's got some
examples to make it easy to use.
But we ended up using the downstreamrepo that NXP has for that.
The main reason is becausewe take advantage of the NPU
on the MCX microcontroller.
(10:05):
So we have a fork of TensorFlowLite Micro that accelerates the
machine learning on that NPU.
When we created this demo, it was veryeasy to just update the west manifest
and select that we wanted this downstreamport from NXP, and it gave us instantly
this ability to accelerate on the NPU.
Excellent.
(10:25):
That explains it to me.
'cause I was wondering the differentiationto the customer when they go to upstream.
NXP fully supports the upstream, butas you just mentioned, when there are
some NXP specific IP blocks or supportthat is not currently in the upstream
repo, like taking advantage of ourNeural Processing Unit, it definitely
(10:46):
makes the downstream nice to have.
You can quickly get thesedemonstrations out as you work to
integrate it with the upstream.
You don't have to wait.
I know we use the downstream Derek forsome new product introductions that
haven't made it to the Zephyr stream yet.
We release them in our downstream andas soon as they're readily available
on the upstream we get them into there.
(11:07):
Is that also correct?
Yeah, that's exactly right.
At NXP we, we want to upstream and workwith the upstream as much as we can.
We have, with our Zephyr development,we have an upstream first mentality.
And nearly everythingwe contribute upstream.
That's where it goes.
But we do have a downstream and we havethese various forks around and it's
(11:28):
like you said it's mainly for stuffthat either we can't upstream right
now, maybe 'cause we don't have fullsupport out there, or in the Zephyr
case they have strict licensing rules.
So sometimes when we enable some ofour IP and hardware that we have with
the licenses we have, we're not able tostick 'em in as Zephyr's upstream repos.
(11:49):
So now we have the downstream.
We have full control over that, wherewe can push stuff in there to make
time commitments that we make tocustomers, and then eventually get
it into the upstream when we can.
Perfect.
Thanks for expanding on that a little bit.
I think our listeners will appreciate it.
All right, so Derek I have seenso many facial detection demos.
I'm sure our listeners, those are, yeah,they're evaluating examples and they say,
(12:13):
oh, yet another facial detection demo.
Maybe you can spend a fewminutes to discuss what are you
trying to technically highlight?
You mentioned the NPU acceleration, someof the other software that we provide, the
portability in an i.MX RT versus an MCX.
But can you differentiate this facialdetection demo and the technical
(12:37):
advantages that you think thecustomers should take a look at?
Yeah, and that's a fair question.
Going back to the history of thisdemo, like I said, we were putting
this together at the time that wewere launching the MCX N device,
which was, it had the new NPU on it.
And that was a big featurewe wanted to promote.
To your point, there was alreadya facial detect demo that we had
(12:59):
done for that device to showcasethat hardware and that silicon.
That was a bare metal demo, meaningit, it did not use any RTOS.
So we thought it would be a great ideato take that demo to port it into Zephyr.
So for one thing, it shows the integrationthat we have for machine learning and
Zephyr and being able to use the NPU.
(13:19):
But then we also wantedto show this portability.
We thought it would be a greatidea to contrast the performance
advantage that the NPU gives you.
So we compared it to the RT1060.
which, it is a muchhigher performance device.
You know, the MCX microcontrolleris 150 MHz, M33 core.
(13:40):
The RT1060 runs at 600MHz and it's an M7 core.
But when you look at this demo and it'san apples-to-apples comparison 'cause it's
literally the same code running on both.
The inferencing time on the MCXdevice is 22 milliseconds and it's
five times longer on the RT1060.
(14:01):
Wow.
I think that tells a great story.
I mean, you got a part that's runningfour times faster, but it takes five
times longer to do the same inferencing.
So it does show the advantage of the NPU.
It also shows how we can take thesame app and scale it around because
there are advantages in the RT family.
It does have more integration.
It's got better graphicsand camera interface.
It's got a lot more performancethat I can offer beyond the NPU.
(14:25):
So it makes sense in manyapplications to use that device.
You get a lot of our customersand users out there who, they
don't just have one product.
They make a family of products,they have a portfolio, they have
like a Good, Better, Best modelthat they sell, and they wanna reuse
as much of that code as they can.
And so they're frequently forcedto have to use different hardware
(14:47):
platforms and they don't wanna haveto reinvent the wheel all the time.
So this is a demo showing wherewe have a real world application
on these different platforms.
And beyond the NPU, it also shows there'sa lot of differences under the hood.
This demo uses a camera to look at yourface, and it's got a display where it
shows what the camera sees, and it'll drawrectangles around the faces it detects.
(15:10):
The RT1060 has a trueCSI camera interface.
It's that a high end displayinterface, but it has a lot
of features built into it.
On the MCX side it doesn't have thosesame interfaces, so it, it doesn't
have a camera interface that doesn'thave a true display interface.
So we ended up using, a SmartDMA, it'scalled, it's a special peripheral we
(15:32):
have on that family that we can configureand that became our camera interface.
When we did this integration in Zephyr.
Zephyr has a, a common video driver that'sable to hook up to cameras and it provides
this, you know, same interface, same APIsthat the application could call regardless
of the hardware they're running on.
And in the RT family, we already hadthat video driver available, with some
(15:57):
examples showing it with our camera.
But when we ported this demo to MCX.
We didn't have that driveravailable at the time.
So we ported Zephyr's video driver tothis SmartDMA, but in this application,
that device is limited on RAM.
'cause you got a, a lotgoing on in the app.
(16:18):
You got the machine learning model,you got the display that's got
frame buffers, and then you gotthis camera that you're bringing in.
So what the bare metal applicationhad done was it brought in a partial
frame from the camera into a smallerbuffer and processed it instead
of bringing in the full frame.
That was a way to save RAM.
(16:39):
So when we ported the driver to Zephyr.
The Zephyr APIs didn't havethat capability, at that time.
They only brought in a full frame Buffer.
So NXP contributed this new feature.
This is frequently what we do aswe're enabling our hardware and
adding demos into the Zephyr world,is we also contribute to Zephyr that
(17:01):
helps with everyone using Zephyr.
So in this case we added this abilityto do these partial frame to the
driver, and we got that upstreamed.
Then that enabled us to usethe same Zephyr video drivers
on these different platforms.
Even though under the hood they're manageda lot differently at the hardware level.
So, very interesting.
And I know you mentioned that theFacial Detection is one of the
(17:22):
first applications shared on ACH.
What other applications are youseeing being added in the future?
Is there something you canshare with us to look forward
to sometime in the near future?
Yeah, there's a very cool one coming.
I, I recently heard your episode withEduardo Viramontes, who also referenced
(17:42):
that there's a Doom demo, the videogame Doom, that runs on Zephyr.
It runs on the same MCX N947FRDM board that we have.
That demo's been out as well.
It's very cool.
It's fun to see and it'scoming to Application Code
Hub, so that's pretty exciting.
I think that's gonna be funwanting to get up there.
All right.
Awesome.
I'm looking forward to that.
(18:03):
We'll be sure to drop the links inthe episode notes so our listeners
can explore the demos for themselves.
Sure, sure.
And this has been great Derek,but we've had a few episodes on
Zephyr and I think we would miss anopportunity if we didn't ask you.
You mentioned it's pretty much 110% ofyour job to advocate and support Zephyr.
(18:25):
So could you just in a couplesentences, what's your elevator pitch?
When you talk to somebody and theysay, "Yeah, I've heard about Zephyr.
I don't get it." How do you get peopleexcited about what you're excited about?
Right.
We have a lot of customers atNXP that have moved to Zephyr and
(18:46):
they've made this decision acrossthe board that they wanna use Zephyr.
They're very engaged with us,and there's definitely some
common themes for why they do it.
And NXP, we have our own reasonsas well, but they're very similar.
I've talked about the portability.
I'd say that's probablythe biggest reason.
At NXP, we love it because wedo have this very broad, massive
(19:06):
portfolio that is very diverse.
Leveraging something like Zephyr thathas this very well defined Hardware
Abstraction Layer makes it easier forus to focus on applications and demo
stuff like this that gets our customersthat much closer to production.
And then reuse it across our portfolio.
But our customers need that same typeof portability and scalability as well.
(19:30):
They don't want to spend all theirtime fussing with drivers and stitching
middleware together and everything.
Zephyr does a lot of that for you.
I think the other key thing is justthe big ecosystem that is Zephyr.
I also heard your episode youhad recently with Eli Hughes.
He does a great job pitchingZephyr as an ecosystem.
(19:51):
Sometimes I talk to people who arejust learning about Zephyr and they had
this perception that Zephyr's just thekernel, like some other RTOSes out there.
They think, "I don't need anotherRTOS", but Zephyr's way more than that.
It has this hardware abstraction layer.
It's got a lot of middleware,either directly integrated as
part of the Zephyr project, or theintegrated as third party modules.
(20:16):
So things like USB Stack, ethernetStack, Bluetooth, and Wi-Fi.
Those things are native stacks in Zephyr.
But what I like about Zephyr is thatif there's already valuable open
source software out there that'savailable for middleware for other
features outside what Zephyr has.
(20:37):
If it's well supported and hasgood community engagement, then
Zephyr will just integrate that ininstead of reinventing the wheel.
It makes it very easy to leveragethose packages within Zephyr and
they'll add examples and stuff too.
So out of box you can have examplesfor these different packages.
You can get 'em running onthe hardware you select.
(20:58):
So like we mentioned TensorFlowLite Micro, that's one of them.
LVGL for graphics is anotherone that gets integrated.
There's like Mbed-TLS andPSA crypto for security.
And the list goes on and on.
There's Mcuboot as a boot loader, butthere's something like 70 other repos that
get pulled in if you just go out there andinitialize the upstream Zephyr workspace.
(21:21):
The other aspect of that ecosystemis also the build environment.
Because Zephyr also includes whatthey call the Zephyr SDK, which is
the collection of build tools needed.
It provides this common interface throughWest that gives you the ability to
have the same development environment,the same commands that you can use
across these different platforms.
It's a very cohesive, massive ecosystemthat gives you all these building
(21:45):
blocks and makes it easy to getstarted jump into your application.
Well, Derek, thank youfor the elevator pitch.
We loved having that at the end, anda big thanks for walking us through
how Zephyr's come into life in theApplication Code Hub from facial detection
on the MCX N to what's coming next.
I'm really lookingforward to the Doom demo.
It's clear ACH is making it easier thanever for developers to evaluate real
(22:09):
world applications with NXP hardware.
Yeah.
Thanks for having me.
Well, again, for our audience, whetheryou prefer command line or a wizard
based tool, the Application CodeHub, it's delivering a streamlined
experience for our customers.
Plus it's got all that soliddocumentation Derek talked about,
and it'll get you started with Zephyrand some of these cool applications.
(22:30):
And as Derek mentioned, there's afew new things coming down the pipe.
Check out the links in theepisode notes to explore the
demo and other Zephyr projects.
And as always, like follow andsubscribe to stay up-to-date with the
latest from the EdgeVerse Techcast.
And to our listeners (22:43):
always stay curious, always keep innovating,
and Bridgette and I'll see youon the next EdgeVerse Techcast.
Popular Podcasts
24/7 News: The Latest
The latest news in 4 minutes updated every hour, every day.
The Clay Travis and Buck Sexton Show
The Clay Travis and Buck Sexton Show. Clay Travis and Buck Sexton tackle the biggest stories in news, politics and current events with intelligence and humor. From the border crisis, to the madness of cancel culture and far-left missteps, Clay and Buck guide listeners through the latest headlines and hot topics with fun and entertaining conversations and opinions.
The Charlie Kirk Show
Charlie is America's hardest working grassroots activist who has your inside scoop on the biggest news of the day and what's really going on behind the headlines. The founder of Turning Point USA and one of social media's most engaged personalities, Charlie is on the front lines of America’s culture war, mobilizing hundreds of thousands of students on over 3,500 college and high school campuses across the country, bringing you your daily dose of clarity in a sea of chaos all from his signature no-holds-barred, unapologetically conservative, freedom-loving point of view. You can also watch Charlie Kirk on Salem News Channel