From the Vault: Regression to the Mean

In this classic episode of Stuff to Blow Your Mind, Robert and Joe discuss regression to the mean, a phenomenon that occurs when a random sampling point is an outlier -- with applications to everything from economics and sports to health, religion and disappointing sequels. (originally published 8/3/2021)

See omnystudio.com/listener for privacy information.

Episode Transcript

Speaker 1 (00:05):
Hey, welcome to Stuff to Blow Your Mind. My name
is Robert Lamb and I'm Joe McCormick. And it's Saturday.
Time to go into the vault. This episode originally published
on August three, and it's called Regression to the Mean,
which is about regression to the mean. Uh, it's an
episode about a statistical phenomenon. But don't don't don't run away.

(00:27):
It's actually, I think super interesting and having this idea
in your toolkit really helps you understand the information that
you encounter in the world, uh much better. Absolutely, So
let's dive right in. Welcome to stot to Blow Your
Mind production of My Heart Radio. Hey, welcome to Stuff

(00:55):
to Blow Your Mind. My name is Robert Lamb and
I'm Joe McCormick. And in today this episode, we are
going to be focusing on a topic that is already
something that's very well known to people who are familiar
with quantitative research and statistics, but less known to the
general public. And uh, and I think that's a tragedy
because it's an idea that should really be part of

(01:17):
everybody's basic critical thinking tool kit, no matter what your
job is. And so in order to introduce this concept.
I thought it would be best to start with a
with a direct illustration from the real world of people
reaching incorrect conclusions by not understanding the subject of today's episode.
And so the illustration I want to start with is

(01:38):
an interesting story told by the psychologist Daniel Kaneman that's
about the illusory power of screaming at pilots. Uh So,
the context of the story is that Knemon says he
was giving a lecture about positive reinforcement to a group
of flight instructures. I think this was in the nineteen sixties,

(01:59):
and Kneman was trying to inform them about what he
believed at the time was the best consensus of scientific
research on learning and reinforcement, which was at the time
that if these flight instructors wanted their students to have
the best possible outcomes, they should focus more on praising
the students when they did well, then on chewing them

(02:20):
out when they did something wrong. And Knomon says that
when he finished his talk, one of the flight instructors
that he had been giving this lecture two got up
and tried to dispute him. He said, no, you're wrong.
And so the direct quote economy and gives from the
instructor here is, on many occasions I have praised flight
cadets for clean execution of some aerobatic maneuver, and in general,

(02:43):
when they try it again, they do worse. On the
other hand, I've often screamed at cadets for bad execution,
and in general they do better the next time. So
please don't tell us that reinforcement works and punishment does not,
because the opposite is the case. So you might think
he has a good point here. If you accept that

(03:03):
this flight instructor has had a lot of direct experience
working with students, and you trust him to remember the
relative frequency of these events pretty well, you might assume
that he has a meaningful rebuke to konom In. Here again,
he says that most of the time, after a cadet
does something bad and he screams at them, they do

(03:23):
better the next time, And after a cadet does something
good and he praises them, they actually do worse the
next time. So if he's remembering these experiences correctly, and
he's had a lot of them, it would really seem
like evidence that praise has a negative effect on learning,
maybe by making the student pilots soft and overconfident. Or something,

(03:44):
and getting chewed out is good for skill development. I
think it's quite easy to see the allure of this,
this false conclusion right right, And it's and you can
also easily imagine how you kind of build upon this
with certain loosely backed up you know, folk ideas about
how you encourage people and how people learn, and you
got to stay on and if they if you tell

(04:05):
them they're doing a good job, they'll get lazy, right,
folk wisdom, tough guy mentality. But Koneman saw something different
in this response, and he says that he immediately set
up an experiment on the spot to demonstrate the flaw
in the flight instructors thinking here, so I want to
read from Knomen's description, He says, I immediately arranged a

(04:26):
demonstration in which each participant tossed two coins at a
target behind his back without any feedback. We measured the
distances from the target and could see that those who
had done best the first time had mostly deteriorated on
their second try, and vice versa. But I knew that
this demonstration would not undo the effects of lifelong exposure

(04:48):
to a perverse contingency. So to explain this, this experiment
a little bit better. Right. He has people stand with
their backs to a target so they couldn't see it,
and they would take two a attempts to throw a
coin and hit the target without any feedback of any kind.
So they're not getting praised, they're not getting chewed out, nothing. Uh.
And after staging a number of these, he found again

(05:10):
what he suspected, that the people who were the closest
on the first throw did worse on their second throw,
and the people who were farthest away on their first
throw tended to do better on the second throw. So
what condiment is actually demonstrating here is something that doesn't
really have anything to do with learning or reinforcement, or

(05:31):
really skills or even human psychology. Instead, this demonstration is
showing the effects of chance, luck, and statistics. What he
was showing is the subject we're talking about today, regression
to the mean. Uh. You'll you'll see that phrase a
lot in in scientific literature and in statistics. But if
it helps to put it in more everyday terms, anytime

(05:53):
you see regression to the mean, you can translate it
in your head as trending toward the average, trending toward
the average. So to make the coin tossing illustration even clearer.
Imagine you throw the coin not twice, but that you
throw the coin a hundred times. So you stand there
throwing the coin a hundred times. And then let's say

(06:14):
afterwards you average together the distance from the target across
all a hundred throws, and you'll come up with some
kind of average distance from target. Uh, just to make
up a number for the sake of argument. Common doesn't
give this. But let's say the average distance from the
target across all your throws is nine centimeters. And remember

(06:34):
that you're getting no feedback at all here, so it's
unlikely that you will be getting much better as you
go on. So, given that the average distance from the
target is nine cimeters, if you throw a coin once
and it happens to be two centimeters from the target
so really close, is your next throw likely to be
about the same as that one, better or worse. Obviously,

(06:56):
it is overwhelmingly likely that your next throw will be worse,
just due to chance, probably closer to the average of
nine cimeters away. And the same goes for throws that
are really far off. You throw something three hundred centimeters
off your next random toss just by chance is likely
to be much better, much closer. So simply put, most

(07:18):
of the time, if you're sampling something in a series
over time, if one sample produces an extreme value, the
next one in the series is more likely to be
closer to the average instead of extreme in the same way.
In my experience. Uh, this is this is why it
can sometimes be liberating to start off a game of

(07:40):
bowling with just a disastrous gutter ball, because because I
know that I'm good enough that that's probably not going
to happen twice in a row, but it's definitely going
to happen at some point in the game because I'm
not that good, you know. I put like playing you know,
once a year or even with less frequency these days.
Oh yeah. And also like why I think a lot

(08:01):
of us have intuitions that when you try something for
the first time and you do really good on the
first attempt, that makes you kind of nervous because you
just know you're probably not going to live up to
that repeatedly. Yeah, like if you get if you get
a strike that first time, then that that first um
what is it round? I can't even remember. This is
how one frequently I bowl, Um, the first role. So

(08:22):
the first role first, the first column. You know, So,
the tendency of regression to the mean or or trending
towards the average is pretty obvious when you're dealing with
something like lots of random coin tosses with no feedback,
But it becomes much more obscure when you're dealing with, say,
a more more limited numbers of outcomes. In the series,

(08:44):
you're looking at and introducing possibly influential variables like pilot
skill and instructor feedback. After all, we would expect that
some variables having to do with instructor feedback should have
an effect on pilots ill, right, That's the point of
teaching is to have an effect over time, and after all,
in this one scenario, the conomen describes the the instructor

(09:08):
believed that his verbal abuse of the students was so
motivating that it made them instantly better on the stick.
And you can't necessarily rule that out, but it's unlikely.
I think I'm convinced that regression to the mean could
more easily explain this flight instructor's belief that screaming at
pilots for screw ups made them better at planes, because, again,

(09:30):
on average, even in the absence of any feedback at all.
If a pilot in training executes a maneuver perfectly, the
random fluctuation from one execution to the next will tend
to mean that their next attempt probably won't be as
good as that really good when the last time. And likewise,
if they make a major error totally botch a maneuver,

(09:51):
they're more likely to do better the next time just
by chance. Both of these tendencies are regression towards the mean.
But then Conomon actually draw is a really interesting observation
about about about our psychology and about culture from this fact,
so to quote him directly, this was a joyous moment
in which I understood an important truth about the world.

(10:13):
Because we tend to reward others when they do well
and punish them when they do badly, and because there
is regression to the mean, it is part of the
human condition that we are statistically punished for rewarding others
and rewarded for punishing them. And that was one of
those things that when I read it, I was just like,
oh my god, that's so true. Um, yeah, yeah, And

(10:37):
in this specific instance, it makes me think about the
special effect of reversion to the mean, fallacies on motivating
belief in the effectiveness of of not just screaming at
pilots in this one case, but all kinds of punishment behaviors,
for example, corporal punishment. Thankfully you hear this less often
these days, but I remember when I was younger, I

(10:58):
used to hear people who would defice end the parental
practice of spanking children by saying, you know, I don't
I don't care what the site scientists say. I don't
care what the research says. I know from experience that
it works. To the extent that comments like this were
based on any real experience and observation and not just
sort of a free form, self justifying statement that had

(11:18):
nothing to do with experience. I bet a lot of
it was fallacious inference of causation actually based on regression
to the mean, just like in this condiment example. But anyway,
I thought it would be interesting to talk a bit
more about regression to the mean today, because it's one
of those things that, again, once you see it, it's
it's pretty simple, it's actually actually pretty clear, but understanding

(11:41):
it can help you have a better sense of how
good science works and help keep you from drawing hasty
inferences in everyday life. Yeah, because it is it is
interesting how kind of an insidious the results can be
the idea that that again, praise is ultimately punished because
is there's going to be a regression to the mean,

(12:02):
to to to to the mean, and then likewise there
can be this illusion, uh that uh that's screaming at
pilots and so forth is going to be the successful
way to go about things. Um. So yeah, this is
I think this is an important episode to cover because
it's the kind of thing that it's the kind of
tool you kind of need tucked in your back pocket,
even if you're just doing something like like scanning science

(12:25):
headlines on a you know, a news server or social
media message board. Yeah, because of course, understanding regression to
the mean is extremely important in what scientists do when
they design good experiments. If you don't take into account
regression to the mean, you can incorrectly believe you have
discovered some kind of tiger repellent or something. Uh. This

(12:46):
concern plays a huge role in the history of medicine.
It's part of the design of good medical research, or
really any field that seeks to find remedies for problems.
So consider a very basic hypothetical, uh path medicines, say
from a hundred years ago. So you know, you have
you have a foot pain that you've never really had before. Uh,

(13:07):
you know, you want it to go away. So you
go to the store and you buy a bottle of
doctor Field Grades No Fail Pantasy for tumors, ulcers, cramps,
and rooms, and you you pull the cork out, you
chug it, and then the next day your foot feels better.
Now you can conclude from this that the doctor Field
Grades cured you. But how do you know actually that

(13:28):
the feelings in your foot didn't just regress to the
mean because the average is a low amount or no
amount of foot pain. And if you don't have a
medication that's tested with control groups and and randomized allocation
into the groups, then how do you know that that
the medicine actually did anything at all? Yeah? Yeah, So
many of the examples you see for this and the applications,

(13:50):
you're dealing with some sort of situation in the world,
whether there is fluctuation and or change happening, often separately
from whatever is being tested. So in this case, yeah,
the doctor Field grads could have just been like just water.
It just just you know, but there is the illusion
that it worked because things got better. But if you
don't have a control group and to you know, to

(14:11):
drive home what that is. That would be like if
you had a had like three different groups and a
study of doctor Field Greats elixir. Here, one group was
taking doctor Field graades elixer, another group was taking I
don't know, let's say a half dose of Feel Grade
or maybe a competitor's tonic. And in one group the
control group was taking nothing was or was taking you know,

(14:34):
just water or something to that effect, something completely innate. Uh.
And that would be that would be a group that
you would judge the results of the other categories by, right,
and you would need to randomly sort the people into
those groups. So it wasn't just that, you know, the
only the people with real severe foot pain we're taking
the doctor Field grades because the more extreme their pain

(14:57):
to begin with, probably the more likely they are are
to have that pain be lessened or go away over time,
just naturally. Right. And uh. And I'm going to have
a more specific example of this a little later in
the podcast. So if you if you still don't get it,
just hang on we'll we'll have another example in a
bit thank. I was looking at an article in the

(15:21):
British Medical Journal from nineteen that was just a collection
of different examples of regression to the mean in real
life medical research. This was by J. Martin Bland and
Douglas J. Altman called statistics notes some examples of regression
towards the mean, and they point out a very common
type of example. So this will be similar to what

(15:42):
we just talked about. The author's right. In clinical practice,
there are many measurements such as weight, serum, cholesterol concentration,
or blood pressure, for which particularly high or low values
are signs of underlying disease or risk factors for disease.
People with extreme values of the measurements such as high
blood pressure may be treated to bring their values closer

(16:05):
to the mean. If they are measured again, we will
observe that the mean of the extreme group is now
closer to the mean of the whole population. That is,
it is reduced. This should not be interpreted as showing
the effect of the treatment. Even if subjects are not treated,
the mean blood pressure will go down owing to regression
towards the means. So again something starts with an extreme

(16:27):
value in certain types of cases, you would just expect
it to have a less extreme value the next time
due to random fluctuation. Uh So again, you know this
could fill you with despair because you might wonder, well,
then how could you ever know if a treatment was
effective or not. But again, this is where the standard
practices of science based medicine come to play. Instead of

(16:49):
just taking people with some extreme measurement and giving them
a treatment, you randomize them into test groups and control groups,
like we were just talking about. So if you have
a large enough sample, you really randomize the groups. People
with the extreme starting conditions will somewhat regress towards the mean,
but they will all regress toward the mean on average
the same rate, whether they're receiving a real potential treatment

(17:12):
or they're in the placebo group. But if the treatment
actually does something helpful, this effect will manifest as the
difference between the two groups. So good scientific research, good
medical research has methods for excluding the effects of reversion
to the mean on their findings. We have the tools,
but we can still fall into the trap of regression

(17:32):
to the mean fallacies, especially in our day to day lives.
Drawing inferences the way that that the pilot and Inconomens
story did, or or even in science if we're not
careful and deliberate about designing experiments. And in addition to
just a methodology design that has you know, a randomized
groups and control groups, there are also ways of trying

(17:53):
to counteract regression to the mean, just through statistical methods
that are maybe less reliable, but there are statistical methods
people can used to try to apply sort of modifiers
to data in order to estimate regression to the mean
and uh and counteract its effects. So again, we have
tools within scientific research to to figure this out, and

(18:13):
it's a lot of what science does is trying to
sort out the difference between regression to the mean and
actual effects of interventions. But in our day to day lives,
we still fall for regression to the mean fallacies all
the time. Yeah, and it's important to realize too that
it's not just a situation where regression towards the mean
could create an illusion of something working when it doesn't. Uh.

(18:36):
You know, sometimes it can just potentially overstate um the
effects of something. For an example of that that I
was looking at was that regression towards the mean, or
the failure to account for it can also overstate the
effectiveness of something like traffic light cameras. Is it making
a difference and cutting down on accidents? Perhaps, but any

(18:57):
actual effectiveness could potentially be overstated by failure to account
for just regression towards the mean. Oh yeah, so where
do you tend to install things like that? High acts
like problem areas? Right, So, if there's like a stretch
of road that has a lot of problems on people
really speeding a lot there or crashing a lot there,

(19:18):
that might be where you stage the intervention. It's possible
some things like that fluctuate naturally over time in different locations,
and you put the cameras in place, and it could
have an effect, but maybe not as much of an
effect as it looks like it is taking place. Again,
if you don't factor regression towards the mean into the study.

(19:40):
Right now, While our TM is a very important phenomenon
to understand and take into account, it certainly doesn't apply
to every sequence of values you could repeatedly sample, so
you also have to be careful not to apply it
in situations where it isn't warranted. I was you know,
there are a million examples. You could cite one that
came to my mind as the orbital decay of a satellite.

(20:03):
Let's say you've got a communication satellite in lower orbit
and you get a reading on its altitude and the
reading is lower than the satellites average altitude. Uh. Now
you might say, hey, I think this means we need
to program a reboost to insert it back into the
orbit where it's supposed to be. And somebody could erroneously
apply regression to the mean here and say, nah, we

(20:26):
don't need to do that. The satellite might just return
to its average altitude. It doesn't apply in this scenario,
even though you are taking repeated measurements of a value
over time, because we know things about the physical characteristics
determining the orbit of satellites and in lower th orbit uh,
and that due to factors like atmospheric drag, their altitude

(20:46):
tends to trend steadily downward over time in a consistent
direction down down, down, So eventually you will need a
reboost in order to put it back up to the
correct distance. So regression to the mean apply is to
certain kinds of data that are repeatedly sampled data where
there is natural random fluctuation back and forth, not a

(21:09):
steady trend in the data in one direction on the
relevant time scale. The other thing that's important to understand
is that systems where you expect to find regression to
the mean are systems in which the repeated data values
you're sampling are to some degree determined by luck or chance.
If a series of values is influenced almost entirely by

(21:31):
deterministic influence, like in the satellite example, by like the
laws of physics, or by some extremely reliable skill with
little room for variation, values don't really regress towards the
mean in the same way because there's just less random
fluctuation back and forth to begin with. The more chance
and random variation plays a role in the outcome, the

(21:53):
more you will tend to observe regression towards the mean
after an extreme sample in in whatever it is you're
looking at, I've I've read that the progression towards the
mean is is not to be confused with the law
of large numbers. For example, uh. This is the the
law that that states, as a sample size becomes larger,
the sample mean gets closer to the expected value. So

(22:15):
a coin flipping example is key here. Flip a coin
and the random results are going to ultimately average out
to a point five proportion. But if you only flip
the coin ten times, you might not see this breakdown. Um.
And this also applies to say, even odds on the
rolling of a of a D six of a six
sided die. Uh So for example, two regular people, that's

(22:38):
just to die that nerves like us, it's a D six. Yeah.
D six is what I could get my hands on.
Because I was like, well, I'm gonna do an example.
I'm gonna try it myself. So while I was putting
together notes for this, I went ahead and rolled ten times,
and I got even even odd even odd even even
even even odd. So that's that's seven to three in
favor of even. So it might make you wonder, well,

(23:00):
is this die broken? Does this D six need to
go away? Because it can't be trusted to roll? Uh?
You know a balanced array of odd and even numbers. Well, no,
that's not the case. Uh. And if I were to
roll this, say another hundred times, another thousand times, I
would see things even out even more to where we

(23:20):
would see this, uh, this point five proportion of odd
versus even right. So these are not exactly the same thing,
regression to the mean and the law of large numbers,
but they are closely related. Both observations require you to
think about statistical tendencies over time, over a time period
of repeated sampling, and both are premised on the knowledge

(23:42):
that repeated samples will tend towards the average. But regression
to the mean has to do with the idea that
if you start with an extreme observation and there is
some role of chance or luck in determining the value
of this observation, the next time you sample it, it's
more likely to be closer to the average. The law
of large numbers is that if in the real world,

(24:04):
the more times you run something, the closer your outcomes
in the real world will will be to the sort
of perfect mathematical average that you would estimate just given
the chances to begin with. Now, I want to come
back to regression towards the mean in um in medical
studies because I found a really interesting one that came
out earlier this year. Uh So, a lot of a

(24:24):
lot of the examples you find involving regression to the
mean involved sports or economics, and I found. This one
discussed in a New York Times article again from earlier
this year titled Intense strength training does not ease knee pain,
study finds by Gina Colada. Uh, this is referring to
a study published in Jama that entailed an eighteen month

(24:45):
clinical trial involving three d and seventy seven participants. Okay, okay,
So the basic situation, the setup for this paper is
that a lot of people have knee osteoarthritis, and one
of the go to treatment recommendations has long been and
strength training. So in this study they decided to look
into it with three basic groups, one that received intense

(25:08):
strength training, another that received moderate strength training, and another
that received counseling on healthy living. So that third group,
that's the control group, they did not have any amount
of strength training, just uh, you know, some positive counseling
about healthy living. Sure, so the researchers here apparently actually
expected to see the intense strength training take the lead

(25:30):
that they were looking to identify what has been just
sort of accepted wisdom, um and and again this this
has been the predominant treatment idea. But instead they found
that the results were the same for all three groups quote,
everyone reported slightly less pain, including those who had received

(25:50):
only counseling. Now why is that? Well, as Colotta points out,
there's there's always room for other effects, especially say the
placebo effect. Uh but regression to the is also a
heavy consideration here and certainly could work in congress with
the placebo effect. Right, So, you don't necessarily have to
assume that the counseling actually helped to heal people's knees,

(26:11):
though it may have in in in some it may
have had some kind of mechanistic effect in some way,
a mind body kind of thing. But you would also
just expect over time, people who have an extreme starting position,
who were starting with a lot of knee pain, to
get gradually better over time. Yeah, so a Colatta rights
quote are the right As symptoms tend to surge and subside,

(26:32):
and people tend to seek out treatments when the pain
is at its peak, when it declines, as it would
have anyway, they ascribed the improvement to the treatment. Uh.
So you know this would this would roughly equate to
yelling at your knee when it's in pain, and it
really make it certainly relates to many other health scenarios
as well various medications and even things like prayer and

(26:53):
you know, supernatural um treatments and attempts to to deal
with pain, et cetera. Yeah, I mean it could apply
to any intervention that is aimed at influencing something that
is naturally variable on its own, right. Yeah, and you
know something that's again any kind of system in which
change occurs when fluctuation occurs. Uh, you know, you can

(27:14):
you can see this applying to not only physical pain,
but also uh, emotional distress, things of that nature, you know.
So again, I think this is an important tool to
have in our our logic tool kit. Now there are
even cases where I'm tempted to think about the application

(27:38):
of regression to the mean, but but where it's probably
a lot harder to quantify exactly what the effects are.
It's cases where it can be difficult to separate out,
say the effects of some kind of deterministic influence like
skill versus how how strong the effect of chance or
luck is. But I think about things even in the

(27:59):
world of the like I think about, you know, the
sophomore album by by a band that has like a
really stellar debut album. Uh, you know, often that is
perceived is disappointing, and you have to wonder, like, Okay,
is it is that often true? Because I don't know
if people get famous and it goes to their heads

(28:19):
and then they you know, they get full of themselves
and make something dumb, or is it because when somebody
has a debut album that's really well received to some extent,
it's so good partially because of luck or chance, and
that's an outlier that you're as you're starting sample yeah, yeah,
And certainly this is an area that's there's a lot
more subjectivity here and and so it's not the kind

(28:40):
of thing you can necessarily have a control group for anything.
But but I think it is quite interesting. And I
did find as I was looking around for some jazzy
or examples or possible examples of aggression to the mean, um,
I found one that that actually gets into a little
bit into the idea of you know, the first and
second album. But also uh, the idea of follow up

(29:02):
films and Hollywood sequels has pointed out both good Yeah.
Has pointed out by Joanna Deong in two thou eighteen
on the blogs scientifically sound movie sequels are potentially a
great example of aggression to the mean. Quote, Hollywood sequels
are only made if the original film is a quote
unquote high quality success. But the average quality of sequels

(29:25):
will be closer to the mean than average quality of
originals of sequels because of regression to the means, So
sequels tend to be of lower quality than the original.
Now I might somewhat dispute the premise here that Hollywood
sequels are only made to films that are high quality
to begin with. Um, But but I still think this

(29:46):
is onto something because there is a movie that gets
a sequel tends to have something about it, something that
people are responding to, whether it's a movie that I
would like or not. Right, I mean, so sometimes obviously
the situation is the film just made a lot of mine.
I mean, I guess that's the key thing. It didn't
make a lot of money. If so, producers are going
to be more inclined to say, let's do that again,

(30:08):
Let's have that experience again of all that money coming in.
And sometimes this this certainly matches up with a quality film.
You have something that really captures people's imagination and it
is of high quality and uh and you know, so
it's really firing on all cylinders. But you know, and yes,
certainly in some cases it's just the right film at

(30:29):
the right time. Or or maybe it has nothing to
do with the film itself. Maybe it's who's in it,
or I don't know what's going on in the zeitgeist
during that particular era. Well, the way I would think
about this is, and I think again, this is onto something.
It highlights that when we experience confusion where we say, like, wow,
you know, the Exorcist is such a great horror movie

(30:49):
and The Exorcist Too is so bad? How could that
be the case? You know, why is it? Why is
such a bad sequel to such a great movie? It's
because of the compare a son of the original to
the sequel that we're experiencing this confusion. Another way you
could just look at it is most horror movies are
direc most movies are bad, and it is only by

(31:13):
comparing the The Exorcist Too to The Exorcist that you
notice this steep drop off. Where another way of looking
at it is that The Exorcist Too is bad like
most horror movies are, and the first one was an outlier.
At the beginning, it was a first film in a
series that happened to be really good A cut above. Yeah, absolutely, like, yeah,

(31:34):
I think this is the correct way to look at it,
and also keeping in mind that just how amazing it
is that any film gets completed, like even a bad film,
Like a lot of people probably worked pretty hard to
make that happen, even if the end results don't really
please anyone at all. But but yeah, I think this
is also an interesting inversion of the opening example of
yelling at pilots as well, because most of the time,

(31:56):
if a flawed movie comes out, people are not clamoring
for the sequel. Um Sequels are rarely guaranteed, so you're
not often going to hear things like, oh, well, that
wasn't great. I hope the next one is an improvement.
I mean some people say that, some people I've said
things like that before, where it will be like, oh,
really flawed film, but maybe there's like a cool idea

(32:16):
I kind of wish it would they would remake it,
even though there's no like logical reason that there would
be like a there would be money behind that idea. Well,
I guess it's kind of different when you're talking about
a one off creative project versus something. I mean, we
live in a kind of different era now because we
were at the height of this you know, cinematic universe
thing with a huge number of the big budget movies

(32:41):
that come out. The big event movies are not one
off creative products, but they are a product that exists
within some kind of franchise or universe or something. So
you just know automatically that there's gonna be another one,
whether this one is good or not. Yeah, like either
it's an established film universe where like you know, they
put out another Marvel movie and it's just terrible, Well,

(33:03):
obviously there's enough momentum. They're not going to stop. They're
not gonna be like, oh, well, less and learned, Well
we'll stop then. No, No, there's gonna be another. Another
example of this might be a successful franchise in another medium,
say a book series, so like the Harry Potter books
for example, or I don't know, Lord of the Rings,
where you know that once they make the Fellowship of
the Rings, there's going to be a follow up. They're

(33:25):
gonna do another one. So in these ways, unless it's
the seventies and it's uh, Lord of the Rings movie
that that ends with Helm's Deep. Well, but they picked
that up eventually. But kay, but but yeah, probably the
Harry Potter films are a better example. And there may
be spe specific you know, things about how that wasn't

(33:46):
guaranteed either. Uh, you know, the economic reality can always
come into play. But for the most part, like those
were when when that started, you knew they were gonna
keep making these at least they were going to make
a follow up, so you could have comments like, well
there were that was just on of flawed in some
of the some of its execution. I hope that they
fix that in the next film. For the most part, yeah,
with one offs, this is not the case. It's like,

(34:07):
if if this film fizzles, then only you know a
few like rare people are going to be clamoring for
a sequel or dreaming about what the sequel would be. Yeah,
I think this observation, but regression to the mean and
movie sequels is actually very on point, but more so
for the films of yester Year, where the more the
more common thing was you'd have a an independent sort

(34:29):
of creative product that it's its own thing, and then
if it resonated with somebody, if it did well, there
would be sequels. I think it's a little it applies
a little bit less today when there's just you know,
we're in the world of franchises and extended universes and
there's just sort of like a guaranteed, ongoing uh conveyor
belt of of new stuff within the Marvel world or

(34:50):
the Star Wars world or whatever. Yeah, but I think
it it is a worthwhile way to think about creative
the creative process, and you know, as opposed to some
of these alternate sort of folk wisdomy ways of thinking
about it. For example, on Weird House Cinema, we recently
talked about Toby Hooper. Toby Hooper is one of those
directors who's often you'll often you'll see descriptions. I think

(35:10):
we've even read part of a review where they they
really they talk about, oh, well, you know he put
out Texas Chainsaw Masacre directed that film and this was great.
It was, you know, just a real lightning bolt um
to the cinematic world into horror itself as a genre.
And then the idea that well he was never able
to capture that magic again. You know that his his

(35:32):
career was just like one long slide after that, which
I don't think is a fair assessment, especially if you
employ regression to the mean, you know, the idea being that, yeah,
he did kind of get lightning in a bottle with that,
with that first big film, that that he was able
to to really bring something together that is an outlier, um,

(35:53):
but that that that's just going to happen. That's just
the way these things work, right, So most movies aren't
that good, So you of the random chance of like
how how good his ideas and execution are from one
year to the next is going to set in and
you might have a different idea about his career if
you were to say, like randomly chronologically reorder all his movies, right,

(36:14):
you know, like if you were to put the worst
ones earlier on or something, people might feel differently about it. Yeah,
well then they would talk about, well, okay, TCM was
peak Toby Hooper, like this was his peak output. Because
this is the kind of the kind of view of
an artist's you know, creative trajectory that we tend to
want to um to follow along, you know, because it's

(36:35):
more story shaped, the idea of assent and then eventually
decent that there's gonna be uh, there's gonna be a
period of high noon in their creative out output, and
sometimes that does match up with the reality. But I
don't know. Even then, we I think we tend to
overlook the dogs in the filmographies of people we love,
you know. Oh yeah, uh, But then again, I mean,

(36:56):
this is interesting because in talking about regression to the
mean applying to creative products like movies, we are acknowledging
that the creative process is not purely a product of
talent and skill, that there is a significant amount of
chance and luck involved in something like how good a
movie turns out to be? Um, And it's hard to

(37:16):
know exactly how to like how to picture that influence
of chance and luck, you know, like, what what is
that in the creative process? It's obviously true because there
are people who can be incredibly skilled in one instance
and then I don't know, things just don't go right
the next time, and to make something that nobody really likes.
But uh, but that's that's just not often how people

(37:37):
like to think about creative talents, and people like to
think about the creative process like it is much more
strictly deterministic. Yeah yeah, or or you look at things
like the Star Wars films, and you kind of like
fall into this idea of thinking this is stuff that
is mind out of the mythic earth, and you know,
it just makes sense that things would accumulate and get better.

(37:58):
So um, but really looking back on it, especially if
you actually like watch documentaries, and there's some great ones
about the production of those films, like it's it's amazing
that Star Wars, the first one in New Hope was
as good as it was, and then it's nothing short
of I mean, it's it's just a pure miracle that
the second one was so much better and like really

(38:19):
nailed it. Like if if the second film had had floundered,
I mean, just imagine how different the cinematical landscape would
have been for decades to come. Yeah, So it's it's
amazing if the first film in a series is good,
and it's super amazing if the second one is good.
And and this is why I think we often find
too that if if part one in part two of

(38:42):
something are of high quality, then you've got to look
out for that part three because that part three, that
part three may be coming to get you. But likewise,
if a part two is rubbish, um, you know, subjectively,
then then part three might pick it up and uh
and get things back on track. So you certainly see
that that kind of fluctuation as well. I have a
question I actually don't know the answer to, but this

(39:04):
would be interesting in terms of I don't know the
high performing output, whether that is in whether that is
a creative endeavor like you know, writing books or creating movies,
or whether that's something even like athletics, like athletic performance,
do you expect to see more random fluctuation in the
performance of collaborative output versus individual output? So say, um,

(39:29):
do you expect more influence of random chance and fluctuation
in the quality of uh books written by a single
author versus you know, movies that have the input of
hundreds of thousands of people? Uh? Or in in the
realm of say sports, like do you expect more random
variation in the output of an individual athletes like you know,

(39:50):
an individual gymnast or something, or in team sports? Yeah?
I could see it going both ways, because yeah, if
you think too hard to about even just like the
film and aology, you can easily get into discussions of
like okay, well is it the same cast and crew
that are producing the sequel. Uh, you know, what happens
when the budget is different, what happens when there are
other constraints, what happens when suddenly there are a whole

(40:11):
bunch of producers that have their ideas about what things
should be. I mean, there's so many different factors to
take into place. Uh. You know, with this example that
you know, perhaps doesn't bear too close of scrutiny, but
but but it's but it's still I think serves as
a nice um illustration of the overall trend that we're
talking about here. Well, it does bring up the fact
that since I mentioned athletes that you know, I don't

(40:33):
know a lot about sports. I'm not a big sports fan.
But but clearly, but regression to the mean is something
that has widely been applied to the world of sports. Uh.
For example, in the observation that often after having a
really stellar season, either an individual athlete or a sports
team will be perceived to underperform the next season. And again,

(40:54):
that very well could have something to do with regression
to the mean. Like, you know, the fact that they're
observed having in a using season is actually an outlier.
You're starting your expectations then and saying like, Okay, now
they're going to be the best forever. Just by random
fluctuation over time, you would expect their next season to
probably be not as good as the first. I wonder

(41:14):
to what an extent this can be applied to, say,
the world of the culinary arts, or even just like
various food crops, like say the selecting a cantalope at
the grocery store, that sort of thing. I mean, I
guess it would apply to pretty much anything where you're
sampling in a series over time, there's plenty of random
fluctuation in what you're sampling, and the first thing you

(41:36):
sample is an outlier in some way really good or
really bad. If those things hold true, then you can
probably expect you're going to see some regression one way
or the other. Yeah. Yeah. By the way, I was
looking around for like really stellar examples of a sequel
film that is widely believed to be uh rubbish, and
I think The Exorcist Too is the primary example. Like

(41:59):
you get into some of the other examples that pop up,
I feel like there's room for disagreement. Um. For instance,
Texas Chainsaw Masacre two is one which I saw popping
up on some of these lists for disappointing sequels. But
I think that's entirely based on who you ask. I
think if you ask us, we will agree that that
that t c M Two is is actually a great film.

(42:19):
It's different from the first one, perhaps if you go
into if you go into part two with the expectations
you had for part one, you may see it as
a dip in quality. But depending on what else you're
bringing to the table, you might see it as an
increase in in quality, or at least or something that
maybe is different but on par with the original. I mean,
it's certainly not for everybody. I mean, it is a
It is a gross, disgusting film in in a way

(42:41):
like the first one, probably even grosser, but also a
sort of satirical masterpiece. Um but I just had another
thought when you said that The Exorcist Too is regarded
as like one of the best examples of a sequel.
That's really rubbish. I mean, it makes me also wonder
about the pretty high estimation critics generally have of the
exer Is three. Makes me wonder if the effect of

(43:02):
The Exorcist to being so bad actually makes people sort
of over. You know, they're like they're ready to be
impressed by the Exorcist three. Yeah. Yeah, I wonder if
that's the case too with it with especially when you
have a situation with the part three coming back and
restoring uh some I don't know, some level of quality
to a franchise. I mean there's also like the Star

(43:24):
Trek example, right, I mean that was long the Long
held up as an example of like, okay, you have
you even Star Treks and your odd Star Treks, right, uh.
And I think you've made a similar case for the
Faster and Furious movies, right, I mean, once you get
to a certain point in the series, I think it's
pretty much all uh, you know, a nitrous boosted brain.
It's it gets you know, it's all like we're driving

(43:47):
cars in space now and flying and all that. But um,
but for the earlier ones, yeah, I'd say the odd
ones are better. Like, uh, three is the first one
where it really starts getting ludicrously weird. Four is kind
of a uh and then five starts. Five is when
the rock shows up, and then but by seven year

(44:07):
golden all right, well we're gonna go ahead and close
this one out here. But we'd obviously love to hear
from everyone about this about regression towards the mean, just
in our daily lives, in various scientific studies. Perhaps you
have thoughts about how this applies to something we've discussed
on the show in the past, because I know we've

(44:27):
we've mentioned regression to the mean in passing before, but
certainly we've never taken the opportunity to really dive into
it and explain it like we did today. Yeah, I
know it's come up in passing, just in us making
comments here and there about like the importance of of
randomized trials and control groups and all that. In the meantime,
if you would like to listen to other episodes of
Stuff to Blow Your Mind, you will find them wherever

(44:50):
you get your podcast. Just look for the Stuff to
Blow your Mind podcast feed. We have our core episodes
on Tuesdays and Thursdays, Artifact episodes on Wednesday, listener mail
on Monday's. On Fridays, we do a little bit of
a weird house cinema. That's our times. We just talk
about some sort of a strange film. Uh, and you know,
tease apart what makes it strange? Uh, let's see what

(45:12):
else so yeah, have you go to stuff to Blow
your Mind dot com that will send you to the
I Heart listing for our show, and there's a button
there for a store if you want to click on
that wealth then you can buy some merchandise that has
Stuff to Blow your Mind logos and whatnot on it,
or perhaps weird House cinema logos and whatnot on it,
and you can get mugs, t shirts, all that kind
of stuff. Uh, just for fun, you know, or but

(45:36):
if you want to be all business about it, you
can make it work to get our merch Now, now
this this is just for fun, Like, don't use these
these clothing items to clothe your nakedness. These are just
for fun. This is just purely extra clothing. Okay, this
should not be your core clothing. Huge thanks as always
to our excellent audio producer Seth Nicholas Johnson. If you
would like to get in touch with us with feedback

(45:57):
on this episode or any other to suggest a topic
for the future, just to say hello, you can email
us at contact at stuff to Blow your Mind dot com.
Stuff to Blow Your Mind is production of I Heart Radio.
For more podcasts for My Heart Radio, visit the I

(46:17):
Heart Radio app, Apple podcasts, or wherever you listening to
your favorite shows.