EP20 - Understanding Science Through LLMs? Beware of Generalisation Bias

In this episode, we delve into one interesting findings in the world of AI and science communication. A new study published in Royal Society Open Science, authored by Uwe Peters and Benjamin Chin-Yee, reveals a systematic problem in how large language models summarise scientific research. Even when prompted for accuracy, many LLMs, including the latest versions of ChatGPT, Claude, and DeepSeek, consistently overgeneralise research findings. They take cautious, specific claims and subtly turn them into broad statements that were never actually made in the original papers.

This phenomenon called generalisation bias may not sound alarming at first, but its implications are massive. Imagine a clinical study that finds a treatment is effective in some patients being summarised as effective for all patients. Or nuanced scientific uncertainty being rewritten as confident advice. According to the study, AI-generated summaries are nearly five times more likely to contain these distortions than human-written summaries. And here’s the twist - the newer, more advanced models are often worse offenders than their predecessors.

If you rely on AI tools to digest research, teach, communicate, or make decisions based on scientific evidence, this episode is essential listening. We unpack how and why this bias happens, explore its potential risks for science, education, medicine, and media, and share practical tips for working smarter with LLMs.