Snacks Weekly on Data Science

In this episode, we’ll explore the unique security challenges posed by agentic AI systems and why embedding trust and safety into these systems from the ground up is critical. We’ll review a few key ingredients for building a secure agentic AI future.

For more details, you can refer to the blog, linked here for your reference: https://medium.com/intuit-engineering/owasp-dishes-out-key-ingredients-for-a-secure-agentic-ai-future-be862...

In this episode, we explore how Oda scaled its A/B testing practices alongside its business growth, focusing not only on building a technical platform but also on creating a culture that supports high-quality, reliable experimentation.

For more details, you can refer to their published tech blog, linked here for your reference: https://medium.com/oda-product-tech/odas-online-experimentation-journey-lessons-learned-and-best-practi...

In this episode, we explore how Booking.com tackled the challenge of predicting reservation cancellations in an ever-changing travel landscape. By shifting from a traditional classification model to a survival modeling approach, the team developed more time-sensitive and flexible predictions that better support their business needs and decision-making.

For more details, you can refer to their published tech blog, linked here for...

In this episode, we will explore how Workday tackle the challenge of measuring the cost of GenAI features. We looked at why LLM-powered features require a new approach to cost tracking, and how the team engineered a telemetry-driven system to make those costs visible, actionable, and fair.

For more details, you can refer to their published tech blog, linked here for your reference: https://medium.com/workday-engineering/measuring...

In this episode, we will discuss how Expedia’s recommendation system is designed to handle both standard destination searches and property-specific searches. While traditional ranking models optimize for broad search behavior, Expedia’s team refines their learning-to-rank approach by integrating property similarity, ensuring travelers get recommendations that align with their intent.

For more details, you can refer to their publishe...

In this episode, we will explore why evaluating LLM-based chatbots is critical for businesses, the limitations of traditional evaluation methods, and what could be a good robust evaluation framework covering both search performance and LLM-specific metrics. 
For more details, you can refer to their published tech blog, linked here for your reference: https://medium.com/data-science-at-microsoft/evaluating-llm-based-chatbots-a-co...

In this episode, we will explore how The New York Times balances algorithmic recommendations with editorial judgment. We discuss their business challenge, examine their hybrid content recommendation system, and look at refinements designed to improve the reader experience.

For more details, you can refer to their published tech blog, linked here for your reference: https://open.nytimes.com/how-the-new-york-times-incorporates-editori...

In this episode, we will explore how Airbnb upgraded its conversational AI system, leveraging LLMs in a controlled and predictable way. We will first examine their business needs, highlighting why traditional chatbot-based workflows were no longer sufficient. Then, we will break down their technical solution, which combines structured workflows with AI-powered reasoning, context management, and a guardrail framework. This ensures t...

In this episode, we will explore the importance of the Large Language Model (LLM) and the forces shaping the LLM economy: competition among AI giants, GPU scarcity, and tokens as the new currency. These dynamics drive innovation and challenge businesses to optimize resources and costs strategically.

For more details, you can refer to their published tech blog, linked here for your reference: https://medium.com/wix-engineering/the-em...

In this episode, we will explore why Klaviyo developed its global holdout group feature and how its engineering team overcame the technical challenges. This feature helps Klaviyo’s customers run fair and unbiased experiments across multiple marketing channels, ultimately enhancing the accuracy of their marketing performance insights.

For more details, you can refer to their published tech blog, linked here for your reference: https:...

In this episode, we will explore why code reviews are critical for a fast-growing marketplace like Faire and the challenges that come with scaling them manually as the engineering team expands. We’ll dive into how Large Language Models (LLMs) offer a game-changing solution—automating code reviews by providing instant, context-aware feedback, enforcing coding best practices, and integrating seamlessly into existing development workf...

In this episode, we will explore how Instacart uses data science to optimize its incentive promotions. We will discuss the business challenge, introduce the concept of surrogate indices, and walk through the step-by-step process of building and applying one.

For more details, you can refer to their published tech blog, linked here for your reference: https://tech.instacart.com/instacarts-economics-team-using-surrogate-indices...

In this episode, we will explore Meta’s AI scaling challenges and how the company leverages productivity and efficiency to optimize its computing power. We also discuss how analytics insights help identify active levers to improve AI development.

For more details, you can refer to their published tech blog, linked here for your reference: https://medium.com/@AnalyticsAtMeta/innovation-demands-compute-how-to-enable-ml-producti...

In this episode, we will explore how Coupang integrates Large Language Models (LLMs) to enhance its machine learning ecosystem. We'll break down Coupang’s business model, key machine learning categories, and the role of Foundation Models in improving efficiency and accuracy. Additionally, we'll walk through the LLM development lifecycle, discuss critical infrastructure decisions, and examine strategies for overcoming challe...

In this episode, we will explore how Expedia ranks lodging options to optimize both customer experience and business objectives. We will discuss the business problem—how ranking impacts Expedia’s success and the challenges of hotel recommendations. Then, we will break down the data science solution, covering the objective functions, features/signals, machine learning architectures, and the evaluation of the system.
For more deta...

In this episode, we will explore Thumbtack’s business model and the importance of recommendations in helping professionals grow their businesses. We will share the team’s machine learning solution architecture, which involved building two sub-models and deploying them using offline inference to meet the business needs cost-efficiently. This quick and nimble approach to machine learning demonstrates how technology can effectively so...

In this episode, we will explore Uber’s Model Excellence Scores (MES) framework, a robust system designed to maintain and enhance the quality of machine learning models at scale. We will unpack its core components—indicators, objectives, and agreements—and explain how they work together to ensure model reliability and performance. This framework enables Uber’s ML ecosystem to operate seamlessly and efficiently, driving both innovat...

In this episode, we’ll explore RazorPay, its business, and the critical role customer service plays in its success. We'll dive into how RazorPay revolutionized customer ticket categorization using generative AI. By replacing customer-selected categories with an AI-driven system, they enabled automatic interpretation of ticket details. This approach incorporated pre-processing, prompt engineering, and a robust knowledge base pow...

In this episode, we will explore the foundational concepts of causal analysis, focusing on its two main pillars: causal discovery and causal inference. We will discuss the types of questions these pillars aim to answer and provide illustrations of related methodologies to better clarify their concepts.