In this episode of the Ruby AI Podcast, host Valentino Stoll talks with special guest Kieran, a prominent figure in the Ruby AI space. Kieran recently gave a talk at the San Francisco Ruby Meetup about his new gem, Leva, which focuses on LLM evaluations in Ruby. Kieran discusses his background, his passion for AI and Ruby, as well as his journey in building AI products, including his tool Cora, which helps manage email inboxes by categorizing and summarizing emails using AI. Together, Valentino and Kieran explore the process, challenges, and best practices of creating AI-driven gems and tools in Ruby, the importance of evaluations, and the fun and creative aspects of integrating AI into Ruby on Rails projects.
Mentioned in the show:
- Kieran Klaassen – Ruby developer, creator of Cora and Leva.
- Leva gem – Kieran's LLM evaluation framework for Rails.
- Jumpstart Pro – “is the best Ruby on Rails SaaS template out there”.
- Stepper / Stepper Motor (workflow engine) – a “journey” with steps for background jobs.
- Jaccard Index – A metric for set similarity (|A∩B|/|A∪B|).
- LangSmith – a platform for building production-grade LLM applications.
- Morph LLM – The Fastest Way to Apply AI Edits (4500+ tokens/sec).
- Friday AI Agent – An AI-powered coding agent that handles PRs from start to finish.
- DSPy.rb – Framework for building AI agents and optimizing prompts.
Highlights:
00:00 Introduction and Guest Welcome
00:53 Kieran's Background and AI Journey
01:20 Building AI Tools and the Leva Gem
03:47 Challenges and Best Practices in AI Development
07:16 Evaluations and Real-World Applications
07:36 Community Recognition and Adoption
12:37 Prompt Engineering and Model Testing
22:06 Leveraging AI for Workflow Optimization
28:35 Visualizing Workflows and Tools
31:44 Exploring Hybrid Orchestration Layers
33:15 Debating Deterministic Workflows vs. Agent Flows
34:28 The Fun of Experimenting with AI and Ruby
34:55 Building Gems and Learning Through Creation
40:03 The Value of Rails in AI Development
46:28 Evaluating AI Outputs and Metrics
50:40 Annotation and Continuous Improvement
53:50 Future of AI and Rails Integration
54:54 Closing Thoughts and Recommendations
Popular Podcasts
NFL Daily with Gregg Rosenthal
Gregg Rosenthal and a rotating crew of elite NFL Media co-hosts, including Patrick Claybon, Colleen Wolfe, Steve Wyche, Nick Shook and Jourdan Rodrigue of The Athletic get you caught up daily on all the NFL news and analysis you need to be smarter and funnier than your friends.
On Purpose with Jay Shetty
I’m Jay Shetty host of On Purpose the worlds #1 Mental Health podcast and I’m so grateful you found us. I started this podcast 5 years ago to invite you into conversations and workshops that are designed to help make you happier, healthier and more healed. I believe that when you (yes you) feel seen, heard and understood you’re able to deal with relationship struggles, work challenges and life’s ups and downs with more ease and grace. I interview experts, celebrities, thought leaders and athletes so that we can grow our mindset, build better habits and uncover a side of them we’ve never seen before. New episodes every Monday and Friday. Your support means the world to me and I don’t take it for granted — click the follow button and leave a review to help us spread the love with On Purpose. I can’t wait for you to listen to your first or 500th episode!
Dateline NBC
Current and classic episodes, featuring compelling true-crime mysteries, powerful documentaries and in-depth investigations. Follow now to get the latest episodes of Dateline NBC completely free, or subscribe to Dateline Premium for ad-free listening and exclusive bonus content: DatelinePremium.com