EP30 - How Just 250 Files Can Poison a Large Language Model (LLM)

In this episode of the Professor Insight Podcast, we examine one of the most striking new studies in AI security, titled Poisoning Attacks on LLMs Require a Near-Constant Number of Poison Samples. Conducted by researchers from the UK AI Security Institute, Anthropic, the Alan Turing Institute, and the University of Oxford, this study challenges a long-standing assumption about how large language models can be compromised. The finding is as unsettling as it is important: a handful of poisoned samples can corrupt a model trained on billions of tokens.

Listeners will hear how the research team ran some of the largest pretraining poisoning experiments ever attempted, using models ranging from 600 million to 13 billion parameters. The experiments revealed that as few as 250 manipulated documents could reliably implant hidden “backdoors,” regardless of model size or dataset scale. The episode explains how these backdoors work, why they persist even through fine-tuning, and what it means for AI safety practices that rely on filtering or data scaling to defend against attack.

This episode matters because it highlights a quiet but critical shift in how we must think about AI security. If the number of poisoned examples required for an attack remains constant as models grow, then scaling up will not make systems safer. Instead, the risks expand with the data itself. For anyone working in AI development, governance, or policy, this conversation offers a grounded look at how small vulnerabilities can have large consequences, and what steps the research community is beginning to take to close that gap.