Anand Kannappan (@anandnk24, CEO @PatronusAI) talks about evaluating AI models for hallucinations, managing data quality, automating the process, and optimizing models.
SHOW: 927
SHOW TRANSCRIPT: The Cloudcast #927 Transcript
SHOW VIDEO: https://youtube.com/@TheCloudcastNET
CLOUD NEWS OF THE WEEK: http://bit.ly/cloudcast-cnotw
NEW TO CLOUD? CHECK OUT OUR OTHER PODCAST: "CLOUDCAST BASICS"
SPONSORS:
- [VASION] Vasion Print eliminates the need for print servers by enabling secure, cloud-based printing from any device, anywhere. Get a custom demo to see the difference for yourself.
- [US CLOUD] Cut Enterprise IT Support Costs by 30-50% with US Cloud
SHOW NOTES:
Topic 1 - Welcome to the show, Anand. Give everyone a quick introduction.
Topic 2 - Our topic today is Preventing AI Model Hallucinations. Before we dig into that, I wanted to ask about your time as Lead Data Scientist at Meta. What was it like to be early into that organization, and what did you take away from your time there?
Topic 3 - Ok, let’s dig into model evaluations and hallucinations. Let’s start at the beginning. How do model hallucinations come about?
Topic 4 - When evaluating models for hallucinations, how does a developer or a data scientist know fact from fiction? Due to its size, complexity, and number of parameters, it’s not feasible to simply fact-check and manually verify inputs to outputs. How is this process evaluated and automated with some level of confidence? Additionally, numerous benchmarks are available. What are your thoughts on the usefulness of the benchmarks?
Topic 5 - How does the concept of data quality play into this? How would we know when a model was given insufficient or improper data vs. a hallucination
Topic 6 - We often hear about how frontier models are running out of training data, and increasingly, synthetic data is being used. Does this impact hallucinations in any way?
Topic 7 - The last item I wanted to ask you about, Partonus AI, also pertains to model optimization. Can you explain that process?
FEEDBACK?
- Email: show at the cloudcast dot net
- Bluesky: @cloudcastpod.bsky.social
- Twitter/X: @cloudcastpod
- Instagram: @cloudcastpod
- TikTok: @cloudcastpod
Popular Podcasts
On Purpose with Jay Shetty
I’m Jay Shetty host of On Purpose the worlds #1 Mental Health podcast and I’m so grateful you found us. I started this podcast 5 years ago to invite you into conversations and workshops that are designed to help make you happier, healthier and more healed. I believe that when you (yes you) feel seen, heard and understood you’re able to deal with relationship struggles, work challenges and life’s ups and downs with more ease and grace. I interview experts, celebrities, thought leaders and athletes so that we can grow our mindset, build better habits and uncover a side of them we’ve never seen before. New episodes every Monday and Friday. Your support means the world to me and I don’t take it for granted — click the follow button and leave a review to help us spread the love with On Purpose. I can’t wait for you to listen to your first or 500th episode!
Crime Junkie
Does hearing about a true crime case always leave you scouring the internet for the truth behind the story? Dive into your next mystery with Crime Junkie. Every Monday, join your host Ashley Flowers as she unravels all the details of infamous and underreported true crime cases with her best friend Brit Prawat. From cold cases to missing persons and heroes in our community who seek justice, Crime Junkie is your destination for theories and stories you won’t hear anywhere else. Whether you're a seasoned true crime enthusiast or new to the genre, you'll find yourself on the edge of your seat awaiting a new episode every Monday. If you can never get enough true crime... Congratulations, you’ve found your people. Follow to join a community of Crime Junkies! Crime Junkie is presented by audiochuck Media Company.
Ridiculous History
History is beautiful, brutal and, often, ridiculous. Join Ben Bowlin and Noel Brown as they dive into some of the weirdest stories from across the span of human civilization in Ridiculous History, a podcast by iHeartRadio.