The Behavioral Data Science Podcast

This is the second of the three-part introduction to this season of the Behavioral Data Science Podcast. During this season, Jake and I discuss the intersection of behavior science and data science as they relate to client outcomes in Applied Behavior Analysis service delivery.

In this episode, we discuss the history of outcome measurement in Applied Behavior Analysis, broadly, and the role the data science can play moving forward....

OpenAI and Anthropic have recently released products where users can connect health-related data and receive healthcare advice and guidance.

Jake and David discuss these products, the potential benefits to ease of access to personalized healthcare-related information, and the potential harms from inaccurate (or dangerous) recommendations and data privacy concerns.

At the end of the episode, they each answer the question we all have...

Realistically, you can't build any model of behavior-environment relations if you can't (a) find the data you need and (b) integrate those data into a usable database.

In this episode of The Behavioral Data Science Podcast, we discuss the many considerations and decisions one needs to make. And, we do so by discussing a seven-year-long project Jake has been working on to build a usable database of all open-source articles p...

Crucial to any behavioral data science project is identifying either (a) what behavior you want to analyze and how you'll get the data; or (b) what data you can get and what behaviors those data allow you to analyze well.

In this episode, we chat about these decisions in the context of the literally wild behavior of birds at backyard feeders.

For the interested, here's a link to the backyard ecology dashboard referenced dur...

In each episode of Season 3, we take a claim from a news story, paper, or hot topic, and walk through how a behavioral data scientist would think about it: clarify the question, identify the operant and respondent principles potentially at play, design the data pipelines, choose the models, and turn the resulting insights into data-based behavior-change tools.

In this episode, we take on the claim that "late-night screen ti...

As we close out Season 2 and our emphasis on LLMs, we had the distinct privilege of chatting with Dr. Elizabeth Garrison. She is one of the few people in the world with domain expertise spanning behavior analysis (BCBA) and artificial intelligence (PhD).

In this episode, we reflect on the state of AI research and industry work pre-ChatGPT and post-ChatGPT release, the shift in academic AI research when the transformer architecture ...

Conversations around AI ethics often focus on a suite of incredibly important topics such as data security and privacy, model bias, model transparency, and explainability. However, each time we use large AI models (e.g., diffusion models, LLMs), we reinforce a host of additional potentially unethical practices that are needed to build and maintain these systems.

In this episode, Jake and David discuss some of these unsavory topics,...

"Explainable AI", aka XAI, refers to a suite of techniques to help AI system developers and AI system users understand why inputs to the system resulted in the observed outputs.

Industries such as healthcare, education, and finance require that any system using mathematical models or algorithms to influence the lives of others is transparent and explainable.

In this episode, Jake and David review what XAI is, classical te...

Prompt engineering involves a lot more than simply getting smarter with how you structure the prompts you enter in an LLM browser interface.

Furthermore, a growing body of peer-reviewed research provides us with best practices to improve the accuracy and reliability of LLM outputs for the specific tasks we build systems around.

In this episode, Jake and David review evidence-based best practices for prompt engineering and, important...

Lots of people like to talk about the importance of prompts, context, and what is sent to an LLM. Few discuss the even more important aspect of an LLM-driven system in evaluating its output.

In this episode, we discuss traditional and modern metrics used to evaluate LLM outputs. And, we review the common frameworks for obtaining that feedback.

Though evals are a lot of work (and easy to do poorly), those building (or buying) LLM-dr...