Luke Marsden (@lmarsden, CEO @HelixML) talks about Private GenAI. What is it? Why do you need it? We also discuss integration into CI/CD pipelines, the layers of a Private GenAI Stack, and why most organizations are opting for RAG over fine-tuning LLMs.
SHOW: 943
SHOW TRANSCRIPT: The Cloudcast #943 Transcript
SHOW VIDEO: https://youtube.com/@TheCloudcastNET
NEW TO CLOUD? CHECK OUT OUR OTHER PODCAST: "CLOUDCAST BASICS"
SPONSORS:
- [DoIT] Visit doit.com (that’s d-o-i-t.com) to unlock intent-aware FinOps at scale with DoiT Cloud Intelligence.
- [FCTR] Try FCTR.io (that's F-C-T-R dot io) free for 60 days. Modern security demands modern solutions. Check out Fctr's Tako AI, the first AI agent for Okta, on their website
- [VASION] Vasion Print eliminates the need for print servers by enabling secure, cloud-based printing from any device, anywhere. Get a custom demo to see the difference for yourself.
SHOW NOTES:
Topic 1 - Welcome to the show Luke. Give everyone a brief intro.
Topic 2 - Let’s start with Priavte GenAI. What is it? Why should organizations out there consider it? Why not just use OpenAI GPT’s and fine tune them?
Topic 2a Follow up - Regulatory Compliance - take the opposing forces in the EU for instance to using SaaS based services based in the United States.
Topic 3 - Let’s break down the layers in a typical Private AI stack. I’m seen various ways to represent this such as infrastructure layer, MLOps layer, models, data layer (typically RAG), etc. How do you break up the stack into individual components
Topic 4 - My mind immediately jumps to similarities in the DevOps space. Abstraction layers and components like Docker and containers comes to mind, integration into CI/CD pipelines, etc. I feel like MLOps is it’s own thing with specific tools and workflows. Does this all come together and if so how?
Topic 5 - Also, what does this mean for versioning and lifecycle management of the models and the data?
Topic 6 - We are seeing more and more data pipelines with backed by multiple models, sometimes in multiple locations. How do handle this from both a scheduling and interface standpoint? Is everything hidden behind APIs for instance?
FEEDBACK?
- Email: show at the cloudcast dot net
- Bluesky: @cloudcastpod.bsky.social
- Twitter/X: @cloudcastpod
- Instagram: @cloudcastpod
- TikTok: @cloudcastpod
Popular Podcasts
Fudd Around And Find Out
UConn basketball star Azzi Fudd brings her championship swag to iHeart Women’s Sports with Fudd Around and Find Out, a weekly podcast that takes fans along for the ride as Azzi spends her final year of college trying to reclaim the National Championship and prepare to be a first round WNBA draft pick. Ever wonder what it’s like to be a world-class athlete in the public spotlight while still managing schoolwork, friendships and family time? It’s time to Fudd Around and Find Out!
Crime Junkie
Does hearing about a true crime case always leave you scouring the internet for the truth behind the story? Dive into your next mystery with Crime Junkie. Every Monday, join your host Ashley Flowers as she unravels all the details of infamous and underreported true crime cases with her best friend Brit Prawat. From cold cases to missing persons and heroes in our community who seek justice, Crime Junkie is your destination for theories and stories you won’t hear anywhere else. Whether you're a seasoned true crime enthusiast or new to the genre, you'll find yourself on the edge of your seat awaiting a new episode every Monday. If you can never get enough true crime... Congratulations, you’ve found your people. Follow to join a community of Crime Junkies! Crime Junkie is presented by audiochuck Media Company.
The Breakfast Club
The World's Most Dangerous Morning Show, The Breakfast Club, With DJ Envy, Jess Hilarious, And Charlamagne Tha God!