Sek Chai, CTO and cofounder of Latent AI, joins The Tech Trek to talk about what it actually takes to get AI running on the edge. We explore the real-world constraints of power, compute, and hardware diversity, why an agent-assisted workflow can accelerate MLOps, and how to choose models that are good enough to ship. Sek also breaks down lessons from selling into the federal market and explains why a clear guiding principle beats chasing every shiny opportunity.
Key Takeaways
Edge AI is a different game than the cloud. Power limits, hardware diversity, and deployment realities have to shape the design from day one.
The best model is the smallest one that delivers the capability and latency you need. Bigger isn’t always better.
An AI agent that understands your data, model, and hardware personas can move teams from idea to deployment much faster.
Whether you’re selling to federal or commercial buyers, lead with capability, then meet security and compliance needs.
A strong tenet should guide product direction and market focus more than raw market size.
Timestamped Highlights
00:30 Why edge optimization matters and what Latent AI does
01:09 The messy reality of heterogeneity and power constraints in edge deployments
02:54 Why most edge AI projects never ship and how an agent can change that
05:03 Mapping MLOps personas and tailoring the workflow for each
11:49 Selling to both federal and commercial buyers without losing focus
15:55 Building a company around a tenet rather than chasing every market
Quote of the Episode
“It’s not the model that you’re really chasing after. It’s that capability.”
Pro Tips
Define capability and constraints first—latency, frame rate, and power budget—then pick and optimize the model.
Collect and use telemetry from experiments and deployments to guide model and hardware choices.
If federal markets are in play, bake security and compliance into your early prototypes.
Call to Action
Enjoyed this episode? Follow The Tech Trek, rate us on Apple or Spotify, and share it with someone working on an edge AI project.
Popular Podcasts
Stuff You Should Know
If you've ever wanted to know about champagne, satanism, the Stonewall Uprising, chaos theory, LSD, El Nino, true crime and Rosa Parks, then look no further. Josh and Chuck have you covered.
My Favorite Murder with Karen Kilgariff and Georgia Hardstark
My Favorite Murder is a true crime comedy podcast hosted by Karen Kilgariff and Georgia Hardstark. Each week, Karen and Georgia share compelling true crimes and hometown stories from friends and listeners. Since MFM launched in January of 2016, Karen and Georgia have shared their lifelong interest in true crime and have covered stories of infamous serial killers like the Night Stalker, mysterious cold cases, captivating cults, incredible survivor stories and important events from history like the Tulsa race massacre of 1921. My Favorite Murder is part of the Exactly Right podcast network that provides a platform for bold, creative voices to bring to life provocative, entertaining and relatable stories for audiences everywhere. The Exactly Right roster of podcasts covers a variety of topics including historic true crime, comedic interviews and news, science, pop culture and more. Podcasts on the network include Buried Bones with Kate Winkler Dawson and Paul Holes, That's Messed Up: An SVU Podcast, This Podcast Will Kill You, Bananas and more.
The Joe Rogan Experience
The official podcast of comedian Joe Rogan.