Lucy Letby: The ludic fallacy

In his book The Black Swan, Nassim Nicholas Taleb presents the “ludic fallacy”: the mistake of applying unvarnished theoretical probabilities to real-world scenarios.

Thanks for reading! This post is public so feel free to share it.

Here is his example:

You have a fair coin which, on the last 99 flips, had come up “heads”. Assuming it is a fair coin, what is the probability of it coming up “heads” on the 100th flip?

Professor of statistics: Easy: 50%.

Professional gambler: Come on, chump, it is 100%. What are the odds of it being a fair coin?

We are looking at a highly unusual outcome. The odds of getting it with a fair coin are something like a googol to one against.

That means this did not happen by chance. We can rule out the “null hypothesis”, therefore. It can’t be a truly random event: either the coin is loaded, or the flipper is somehow rigging the game.

This is an effective logical argument:

If:

The coin is fair, andThe flipping method is fair, thenThe coin will not land heads 100 times in a row.

Therefore, if the coin does land heads 100 times in a row, then either:The coin is not fair, orThe flipping method is not fair.

So, we can try to get to the bottom of this.

We can ask to inspect the coin. We can test it out for ourselves. If, on a thorough inspection: its weight, shape, balance, density, magnetism and so on seem fair, and if we flip it a few times for ourselves and it yields expected something like the expected 50/50 outcome, we can rule out the coin as the problem.

We can ask to inspect the flipping process. We will need to be wary: an experienced hustler may be skilled at conjury. She may engineer “heads” each time through misdirection and sleight of hand. Especially if the flipping happens “off-stage”. If we can’t see the coin being flipped, we don’t know how it may have been edited or fixed to omit all the “tails” outcomes, or force the heads result.

In any case, if we are not allowed to inspect the coin, or we are not allowed to observe the flipping method, alarms should sound immediately. This is what a conjurer would do. We should not let ourselves be misdirected here.

Ludic fallacy applied

We have the potential for a “ludic fallacy” in Ms. Letby’s case. We know the probability of a nurse being rostered on for a given shift. Depending on how hard she works, it is between one-in-five and one-in-three. Here, there were 25 “events” — collapses for which she was charged. She was on duty for all of them.

The chances of the same nurse being present at 25 collapses in a row is a bit like flipping 100 heads in a row: the odds against it are not quite a googol to one, but they are still so vanishingly small — in the billions to one against — as to be impossible.

So we can reframe our logical argument:

If:

Ms. Letby is innocent, andThe sampling method for the collapses is fair, thenMs. Letby will not be on duty for 25 collapses in a row.

Therefore, if Ms. Letby was on duty for 25 collapses in a row, then either:She is not innocent, orThe sampling method for the collapses was not fair.

The “null hypothesis” — that the collapses were caused by unrelated natural causes and Ms. Letby’s presence was a pure coincidence — is off the table.

We have two things to consider: firstly, what are the “base rate” probabilities for these alternative explanations, and secondly, what evidence do we have to support the alternative explanations that might change those?

Base rates

What is more likely? That someone, perhaps inadvertently, cherry-picked some statistics, or that a nurse tried to murder 25 premature infants?

I don’t have hard data, but I hope you won’t find the suggestion that people being a bit casual with information is a bit more likely than nurse serial murderer, if you had to choose.

Here, we do have to choose. We should be prepared to change our minds if the evidence requires it, but until we have a good reason, we should assume an unusual coincidence of collapses is more likely to be sampling bias than serial murder. Because, all else being equal, sampling bias is common, and serial murder isn’t.

So is all else equal? There are things we can check.

We can