In this episode, we explore how engineers are embedding powerful AI directly into hardware – no cloud connection required.
Michaël Uyttersprot from Avnet Silica and Cedric Vincent from Tria Technologies reveal how they run ChatGPT-quality language models on resource-constrained embedded devices. What once demanded data centre infrastructure now fits onto chips with just 2GB of RAM.
The conversation covers the technical challenges of cramming billion-parameter models into embedded systems, real-world applications from conference rooms to industrial robotics, and the three compelling reasons driving this shift: data privacy, power efficiency, and cost control.
Michaël and Cedric discuss hardware platforms from AMD, NXP, and Qualcomm, explain techniques like quantisation and mixture of experts, and demonstrate applications including a vintage telephone box that lets you call avatars from different time periods.
Tune in to learn why the future of AI might not be in the cloud at all – and what that means for industries from manufacturing to healthcare.
#AI #LLM #embeddedsystems #IoT #privacy #wetalkiot
Summary of this week's episode:
02:48 What makes large language models special
05:27 Why run LLMs locally on embedded devices
07:42 Real-world applications: Vision LLMs and OCR
11:12 Technical deep dive: How to fit billions of parameters into tiny devices
18:52 Understanding temperature: Making AI creative or accurate
22:41 Industries moving fastest: OCR, security, and robotics
24:52 Future applications: Robotic arms and time series analysis
28:00 The biggest technical hurdle: Power consumption
30:55 Advice for engineers: Start with llama.cpp
Show notes:
Michaël Uyttersprot: https://www.linkedin.com/in/micha%C3%ABl-uyttersprot-aaa971211/
Cedric Vincent: https://www.linkedin.com/in/cedric-vincent-19222910/
Tria Technologies: https://www.tria-technologies.com/
Generative AI at the Edge: https://my.avnet.com/silica/solutions/technologies/artificial-intelligence/generative-ai/
The podcast episode where the generative AI examples where discussed: https://www.podbean.eu/ep/pb-9juiy-d4dec4
How to enhance embedded systems with Generative AI and Local LLMs | Michael Uyttersprot at HWPMAX25: https://www.youtube.com/watch?v=wL9g2wJ1a7c
Listen to the "We Talk IoT" Soundtrack on:
Spotify: https://open.spotify.com/playlist/05MOV4OV2MH2in2txsAGtG?si=ad08112cb8d443f4
YouTube: https://www.youtube.com/watch?v=D-NvQ6VJYtE&list=PLLqgVFfZhDRVYmpEqbgajzDvGL4kACRDp
The Llama song: https://youtu.be/JavZh3y1ue0
About Avnet Silica:
This podcast is brought to you by Avnet Silica — the Engineers of Evolution.
Subscribe to our newsletters here: https://my.avnet.com/silica/resources/newsletter/
You can connect with us on LinkedIn: https://www.linkedin.com/company/silica-an-avnet-company/. Or find us at www.avnet-silica.com.
Popular Podcasts
Las Culturistas with Matt Rogers and Bowen Yang
Ding dong! Join your culture consultants, Matt Rogers and Bowen Yang, on an unforgettable journey into the beating heart of CULTURE. Alongside sizzling special guests, they GET INTO the hottest pop-culture moments of the day and the formative cultural experiences that turned them into Culturistas. Produced by the Big Money Players Network and iHeartRadio.
Crime Junkie
Does hearing about a true crime case always leave you scouring the internet for the truth behind the story? Dive into your next mystery with Crime Junkie. Every Monday, join your host Ashley Flowers as she unravels all the details of infamous and underreported true crime cases with her best friend Brit Prawat. From cold cases to missing persons and heroes in our community who seek justice, Crime Junkie is your destination for theories and stories you won’t hear anywhere else. Whether you're a seasoned true crime enthusiast or new to the genre, you'll find yourself on the edge of your seat awaiting a new episode every Monday. If you can never get enough true crime... Congratulations, you’ve found your people. Follow to join a community of Crime Junkies! Crime Junkie is presented by audiochuck Media Company.
Stuff You Should Know
If you've ever wanted to know about champagne, satanism, the Stonewall Uprising, chaos theory, LSD, El Nino, true crime and Rosa Parks, then look no further. Josh and Chuck have you covered.