Docker launched "Docker Model Runner" to run LLMs through llama.cpp with a single "docker model" command. In this episode Bret details examples and some useful use cases for using this way to run LLMs. He breaks down the internals. How it works, when you should use it or not use it; and, how to get started using Open WebUI for a private ChatGPT-like experience.
🙌 My next course is coming soon! I've opened the waitlist for those wanting to go deep in GitHub Actions for DevOps and AI automation in 2025. I'm so thrilled to announce this course. The waitlist allows you to quickly sign up for some content updates, discounts, and more as I finish building the course. https://learn.bretfisher.com/waitlist🍾
★Topics★
Model Runner Docs
Hub Models
OCI Artifacts
Open WebUI
My Open WebUI Compose file
Creators & Guests
- Cristi Cotovan - Editor
- Beth Fisher - Producer
- Bret Fisher - Host
- (00:00) - Intro
- (02:23) - Model Runner Elevator Pitch
- (03:05) - Enabling Docker Model Runner
- (06:05) - Self Promotion! Is that an ad? For me?
- (06:40) - Downloading Models
- (08:48) - Architectrure of Model Runner
- (12:26) - ORAS
- (12:46) - What's next for Model Runner?
- (13:50) - Troubleshooting
You can also support my free material by subscribing to my YouTube channel and my weekly newsletter at bret.news!
Grab the best coupons for my Docker and Kubernetes courses.
Join my cloud native DevOps community on Discord.
Grab some merch at Bret's Loot Box
Homepage bretfisher.com
Popular Podcasts
Stuff You Should Know
If you've ever wanted to know about champagne, satanism, the Stonewall Uprising, chaos theory, LSD, El Nino, true crime and Rosa Parks, then look no further. Josh and Chuck have you covered.
Dateline NBC
Current and classic episodes, featuring compelling true-crime mysteries, powerful documentaries and in-depth investigations. Follow now to get the latest episodes of Dateline NBC completely free, or subscribe to Dateline Premium for ad-free listening and exclusive bonus content: DatelinePremium.com
CrimeLess: Hillbilly Heist
It’s 1996 in rural North Carolina, and an oddball crew makes history when they pull off America’s third largest cash heist. But it’s all downhill from there. Join host Johnny Knoxville as he unspools a wild and woolly tale about a group of regular ‘ol folks who risked it all for a chance at a better life. CrimeLess: Hillbilly Heist answers the question: what would you do with 17.3 million dollars? The answer includes diamond rings, mansions, velvet Elvis paintings, plus a run for the border, murder-for-hire-plots, and FBI busts.