The Failings of Forensic Science, Part I

What if bad science put you in prison for a crime you didn't commit? How many guilty offenders are still out there because of pseudoscience, fraud or disorganized laboratories? In this episode of Stuff to Blow Your Mind, Joe and Christian review the use of forensic science in law enforcement. Scientific evidence is prevalent in both courtrooms and popular culture. But why are we only recently questioning its validity?

Learn more about your ad-choices at https://www.iheartpodcastnetwork.com

See omnystudio.com/listener for privacy information.

Episode Transcript

Speaker 1 (00:03):
Welcome to Stuff to Blow Your Mind from how Stop
Work dot com. Hey, so, I just wanted to let
you know that Christian and I ended up talking about
this topic for a really long time. So we decided
to split the episode in two, and this is going

(00:25):
to be the first half of our discussion. You can
tune in again next time to hear the second half.
Welcome to Stuff to Blow Your Mind. My name is
Joe McCormick and I'm Christian Sager. So our co host
Robert is away this week. He's on vacation, and the
two of us thought that this would be a good
opportunity to talk about a little side project that we've
been working on for like six months now, longer than that.

(00:46):
Are you talking about that breeder reactor we've been working on?
That's our side side project. This one is is a
little bit more on the books. This is the one
where we've been looking into d I Y for forensics.
Oh yeah, so Christian and I, well may Christian, I
want to give you all the credit because it's true
that you've had this idea of wanting to produce a
video series for How Stuff Works showing how you can

(01:08):
do your own forensic science investigations just like you'd see
Uh well, I don't want to say just like you'd
see on c s I, because it turns out all
all that stuff you see is mostly Faye garbage. Um,
but like a forensic investigator would do it a crime scene,
So so how you can use tools available to you
to figure out what happened if say there's blood spatter

(01:29):
all over a wall, or if there is a fingerprint
left on a surface. And so the first one that
we did was like a demo because we wanted to
like kind of test it as a proof of concept, right,
And we took one of our studios here at how
Stuff works and we covered the walls with paper. Yeah,
we got a mannequin head and put some fake blood
packets on it. Yeah. And then um, we've basically had

(01:51):
Lauren Vogel Baumb from Forward Thinking come in with Thor's
hammer mulenir, which we just have laying around and just
pound on this mannequin's head so that the blood would
splatter as if it was a person being hit with
a real hammer. It's to see how the blood was splatter.
And we were testing basically the premise that you of
how blood splatter analysis works and there's some math to it.

(02:14):
So then we subsequently did the stringing, which they've probably
seen on these forensic TV shows like C S I,
where they string crime scenes and there you're trying to
identify the trajectory of the blood that hit to leave
the stained pattern that you find. Yeah, exactly, so you
can sort of figure out both where the crime happened
and what the like height. And I guess like three

(02:37):
dimensionally is the best way to explain it, right, you
can explain where it happened in the room, but also
the height of the blow. Uh. And so we did
it and it it worked, um sort of. Yeah, it
was a mess with all of the complications we have. Ye. Yeah,
I mean we definitely weren't as prepared for how difficult
it was as I thought. And then we posted some

(02:59):
photos to the stuff to blow your mind social media accounts,
and at least one, maybe two people popped up if
you're listening, thank you. Um they were blood spat spatter
analysis experts, and they gave us some advice. Uh. And
we still we've just been so busy with everything else
that we do here, we just haven't been able to
get to the rest of it. But the idea was

(03:19):
we were going to take this and we're gonna extrapolate
it out into like a four maybe five part series
that was going to be a locked room murder mystery
where each episode Joe and I tried to solve a
crime by doing d I Y forensics in the room.
And we were going to do the blood spatter one,
We're going to do paper chromatography. We were talking about

(03:39):
dusting for fingerprints with super glue, which is really interesting.
We were going to talk about decomposition of bodies and
how do you figure out the time of death. Then
we got into looking at some other stuff. We had
to bite marks, hair and fiber analysis, stuff like that,
and we were like, how valid actually is this? I
mean not not just beyond like us doing the I

(04:00):
Y version of it with like ziplock, Baggies and tweezers
here in our studios, but like, how actually valid is this?
And Josh Clark from Stuff you Should Know started talking
to us about it, and he was like, you know,
there's a lot of really bad pseudo science in this,
and he sent us a great article. This is something
that Josh and I had talked about several times before.
Actually it's sort of a running conversation we have about

(04:23):
problems that keep emerging in forensic science. Absolutely. Yeah, So
he pointed this out to us, and we thought, you
know what, uh, why don't we do an episode of
stuff to blow your mind about this was so we
can sort of get our thoughts clear, really do a
deep dive on the research, and then maybe one day
we'll return to this video series and we'll make sure

(04:43):
that the stuff that we're doing, first of all, is valid.
But that second of all that we can what I'm
hoping to do with it is also somehow within this
locked room narrative that we're going to construct also tell
the audience like you can't rely on hair evidence, for instance,
right or bite marks because there's a lot of subjectivity
to how that's done. Right now, So that's what we're

(05:05):
here to talk to you about today, is the pseudoscience
and the sort of false interpretations and there's there's just
a lot of problems with forensic science, and I think
people within the field that would acknowledge that as well. Yeah,
and it certainly has been openly acknowledged. So one resource
that we're gonna be referencing throughout this episode is this
massive compilation research document put together by the National Research

(05:28):
Council and published in two thousand nine called Strengthening Forensic
Science in the United States. And this was put together
out of a out of a commission I think funded
by Congress to look at the state of forensic science
in the United States and and take a scientific, analytical,
critical approach to it to say how well are we

(05:48):
using forensic science in our courts, Like how well how
scientifically established are the methods that are being used, how
often do they get the right answer as far as
we can tell, Because it has become more and more
clear that lots of traditional methods used in forensic science analysis,
the kind of science we would use to analyze a

(06:09):
crime scene, to identify a suspect, to demonstrate the guilt
of a suspect, etcetera. Uh, these things are in many
cases full of flaws. And I've seen it alleged that
really nuclear DNA analysis is about the only forensic science
discipline used in US courts that isn't thoroughly riddled with problems. Actually,

(06:35):
and we'll talk about it in this episode two. There
are even it can be vulnerable, but I think it's
generally considered the best. It is considered the best, Yeah,
but there are issues with it as well. UM So
I got a question for you. Have you ever served
on a jury before? No, I haven't. I've I've had
you know, jury selection days, but I've never I've never
been picked for a jury. So a couple of years ago,
I was picked for a jury and I don't know

(06:58):
how this happened, but somehow I ended up foreman and uh,
little do they know what power they put in the
hands of this uh, this doom purveyor. It was a
real weird case and I won't take you all down
the long road of it, but I will say you know,
one of the people that came in it was a
it was a cocaine uh possession and distribution case, but
also a firearm possession case for felon who was not

(07:19):
supposed to be carrying firearms. Um. The they had a
cocaine expert come in from their forensics lab. And you know,
like with any rhetorical position, especially in the courtroom, I
mean this is where like a lot of the Greek
terminology for for rhetoric came from. They first started off
by presenting the ethos the quality of the character of

(07:41):
this woman. She gave us her how many years that
she had been working in the lab and in the field,
and what her degrees were, and what her training was,
and how many cases that she had looked at and
all that stuff to sort of establish upfront, this is
a person you should believe, right, And that's pretty much
standard practice when you're in a courtroom and somebody who's
a science expert comes or a forensics expert comes to

(08:04):
testify about a case. But what you don't know is, necessarily,
like the actual field itself, how much solidity there is
to the discipline. Yeah. So imagine you are on a
jury and it is eighteen thirty five or so, so

(08:25):
you are not the scientifically literate person you are today.
You are maybe a farmer who has been called in
for jury duty, and it's a murder trial, and the
prosecution brings on an expert witness to testify to the
guilt of the defendant. And the expert witness who comes
on says, look, here is a map of the different
organs on the human skull. And as you will see

(08:49):
in this sketch of the defendant's head, they have an
extremely pronounced organ of murdering, and that's a bump that's
right here on this part of the head, and the
principles of phrenology, we can demonstrate that this person is
by nature a murderer and will kill again if released.
Now we know that that sounds like complete bunk. And

(09:11):
wouldn't life be super easy if we could just touch
everybody's heads and figure out if there are murderers? It
would be much easier. It would also give you more
excuses when you're touching the heads and people are saying,
why are you doing? What do checking? Make sure? Make
sure I'm safe. I don't want to be in the
presence of somebody with a pronounced organ of destructiveness. Um

(09:32):
but so, yeah, we all know today phrenology is nonsense.
But back then a lot of people would not have
had the knowledge to do that. And it's not because
they were stupid, they just didn't know. It's probably presented
in much the same way. This learned gentleman gets up
in front of you and says, I am an expert,
and all phrenologists agree. Um. So, in that case, if

(09:54):
if the judge has decided that this is admissible testimony
and you're on the jury, how would you know to
quite usenet And then even if you're the judge, the
judge might know not exactly. It's not a qualification to
be a judge to also have a science degree exactly.
Now we are going to talk, believe it or not,
about a real case where somebody tried to use phrenology

(10:15):
in the courtroom in the eighteen hundreds. But we'll get
to that actual historical example in a bit. First we
should back up, I think and and look a little
bit more generally at a sort of top down view
of what are some of the problems with forensic science
as it's used, especially in U. S courts today, but
this is going to apply to a lot of courts
around the world in the general sense. Yeah, and and

(10:37):
like as usual, because we're Americans and we're here in
the US, a lot of our research is focused here.
But the best resource that I was able to find
came from Popular Mechanics, and they had a really good
article on the myths of c s I and they
began by establishing that forensic science as we know it
it was mostly created by police, not by scientists, and

(10:59):
it's based more on common sense rather than scientific practice.
And we all know what common sense is. Common sense
is the reasoning faculty that tells you that the Earth
does not move and is flat. Yeah, b O B.
I'm downe with B O B. Well, I mean those
things are common whitest thing I've ever said on this show.
I'm sorry, Uh yeah, I mean so common sense, as

(11:20):
we know from all kinds of fields of science, very
often betrays us. Common sense is useful for getting your
average stuff done day today, but it is not good
for for deducing truths that are obscure. Yeah. Right, So
this popular mechanics article goes in depth and it says okay,
and it and it points out DNA testing right off
the bat, and they said, DNA testing is is really good,

(11:42):
and it's made it possible for us to re examine
all this other biological evidence that we've taken from past trials,
so that more than two hundred people this is in
the US have had their convictions overturned because the DNA
analysis refutes the other biological evidence that was used in
the case. Fifty percent of these cases involved bad forensic

(12:03):
analysis contributing to their imprisonment, and they refer to this
as the c s I affects of this bringing up
the show. Yeah, the c s I effect, as I've
kind of come to understand, let me know, if this
is in line with what they're saying. As I understand it,
it's where juries tend to expect to encounter scientific style

(12:23):
evidence presented in the courts, or especially DNA evidence. Yeah. Absolutely,
And so it's a perpetuation of the media versions of
the forensic examiners solving difficult cases the science and cutting
edge tech. Right, Like, so when you think of c
s I and you think of, like, wow, this this
is how old I am, Like David Caruso throwing on
his sunglasses and whipping back his red hair, and then

(12:44):
there's like c g I zooming in on a dead body,
and like the explanation exposition about like how forensics works.
That's what they're expecting, right, They're expecting some David Caruso
slick type to show up and explain this to them.
There's an argument though, that this is a misconception that
absolutely sways jurors before they're really even in the courtroom. Yeah. Um,

(13:06):
Like it leads to the misperception that generally the methods
leading to the prosecution, identification and prosecution of a defendant
are very scientific in nature. Not only that, but that
they all cases will have some kind of scientific quote
unquote scientific evidence, uh engaged or as evidence for them

(13:30):
to review. Yeah. So, but what's actually going on is
a little bit more difficult. Like like all things in
the world, America's forensics labs are totally overburdened and understaffed,
and a two thousand five studied by the Department of
Justice found that the average lab has a backlog of
four hundred and one requests. So that's why, Like, for instance,

(13:52):
that case that I was the forem and on, I
think it happened like a year year and a half
after they actually arrested the guy because they were waiting
on evidence. It took forever for them to actually get
to the evidence into Like in this case, they were
looking at the cocaine that was found on his person
to see what percentage of it was pure cocaine. Um.
So this isn't even getting into all of the state, city,

(14:13):
and now federal forensic departments that have been scandalized by
mishandling evidence or just straight up committing fraud. Uh So,
I mean, even outside of the science angle that we're
talking about. There are the pseudoscience angle we're talking about,
there's the mishandling of evidence which makes the science bad,
and then there's just people committing fraud, right, which is well,

(14:35):
I mean unethical. I guess you could encounter fraud in
any situation, like even in a even in a field
where all of the established methodology is well vetted, scientifically
valid methodology, you could still have somebody falsified data, somebody
who has like the quote unquote quality of character to
testify in a courtroom. But whatever, somebody somebody got to them.

(14:59):
But or than just telling people not to be liars,
there's really not as much to do about that. So
I think we're probably not going to focus as much
on fraud today, but it's just important to know that
it's out there, right, especially like next time one of
you is in the jury box. You know, just consider
these things that we're talking about today. Uh So, the

(15:19):
argument basically goes like this, the forensics science field, it
has a foundation that's very shaky and very subjective. In fact,
the American Society of Crime Laboratory Directors says, look, there's
no advanced degree that's required to have a career in forensics.
Some people do, but it's not inquired. I think it

(15:40):
varies field to field because there will be for example,
like maybe like boards or licensing organizations that work in
different fields and they have different requirements. Sure yeah, especially
probably based state to state all that stuff. Um, And
one question I have and let's I don't want you.
I mean, you can answer it right now or we
can think about it and come back to it the
end of the episode. But should it be should they

(16:02):
be required to have an advanced degree? Uh? Well, I
mean I think that's gonna very case to case again.
I think I think methodology is more important than credential ing.
Yeah yeah, Um, Like you can have somebody with the
high school education and if they are rigorously forced to
follow certain methodologies, I would probably be okay, but still

(16:23):
believe it. Yeah. And or computer automated systems that are
forced to follow those rigorous methodologies too, which is something
that people are talking about. So it's been argued that
the body of research behind forensic science is just totally incomplete.
The methodologies aren't precise. Like I said, there's lying in
fraud today. We're mainly going to concentrate though, on whether

(16:46):
the science itself adds up so that you can take
valid conclusions from it and use it as evidence in
a courtroom. Right, pseudo science problems in the scientific method. Yeah,
We've had chemists who have faked results and gone to
prison for drug convictions. There's been sloppy work that's been
done than thousands of cases related to Saint Paul, Minnesota, Detroit, Michigan, Philadelphia,
North Carolina, Houston, and more. I mean, this is like,

(17:07):
this is everywhere, but in two thousand five, this is
this is the what led to the paper that we
were talking about earlier. Congress commissioned the National Academy of
Science to examine the state of forensics in the US
in the U S law enforcement, and they this group
found in two thousand and nine that there were quote
serious deficiencies in the nation's forensics science system, and they

(17:29):
advocated for extensive reform. And they said, apart from d NA,
remember we're gonna come back to DNA later, there is
no single forensic discipline that has been proven with a
degree of certainty to be able to match a piece
of evidence to a suspect. Now that brings up an
important thing that I think may come up again. In

(17:49):
this episode, which is that some fields, as practiced, may
be better able to sometimes exclude the possibility of of
a defendant being gilt rather than matching. If that makes
any one example I know we'll talk about later. His
bite marks. Yeah, and that that's something I've seen pointed out,
is that while it might be hard to say, Okay,

(18:11):
this bite mark identifies that this truly is the defendant,
you could more easily say this makes it clear that
this bite mark was not left by the defendants. Yeah.
It's complicated, right, and it's uh, it brings me back,
as many of our episodes do episodes do lately, to
the wicked problems that Robert and I talked about earlier

(18:32):
this year, that just what's going on with forensics in
the justice system in general is a wicked problem. And
I don't think it's gonna be solved with one answer, right,
but but it's worth looking at. Well. I mean, one
thing we can take comfort in is that this big
two thousand nine report is very extensive, and I hope
that this already has in some cases led to some

(18:53):
initiated reforms, and it will help lead to continued reforms
in the future. So it's not like nobody Reckon noises
this problem and nobody's doing anything about. Yeah, I suspect
that there are a lot of people who are working
very hard right now. You may even be listening to
the show who are like, hey, you know, we're doing
our best over here. So we get that. But but
it's also important to sort of walk through the process

(19:15):
and figure it out. The worst implication of this, though,
Joe is like, this is and and this is extrapolating
it out to. One reason why a lot of people
care about this now is because it's in our pop
culture right Cereal, the biggest podcast out there, Making a
Murderer on Netflix, all of all of which touch upon
these things. There's an implication that if the forensic science
is wrong, that there are innocent people who are jailed

(19:37):
for crimes they didn't commit, and subsequently there are guilty
criminals out there who are still roaming free to so,
um what we're going to talk about this group a
lot as well. The Innocence Project found that when prisoners
were exonerated by d n A, the real perpetrators were
identified in half of those cases, and in all but one,

(19:58):
the real perpetray Waters continued to commit crimes, serious crimes
after the innocent person went to jail. Yeah. So the
Innocence Project, if you haven't heard of it, it's an
advocacy organization, So it does you might say, probably have
an ax to grind in this issue, but but it's
also yeah, it's uh. I think it is abundantly clear
that that many innocent people have been convicted of crimes

(20:21):
they didn't commit. Many guilty people have been let go
on the basis of bad forensic evidence, or or have
been let go simply because somebody else has made to
cop for the crime that they committed. Um, and yeah,
and these people so so somebody who actually did a
murder is out roaming the streets because somebody else is
in prison for it. Yeah, And that's just one disappointing

(20:43):
but too terrifying, right, uh. And so another part of
the problem. And I don't think we're gonna have a
lot of time to dive deep into this today. But
it's also important to remember that the legal sciences don't
get as much federal funding as other disciplines when it
comes to research like this, so subsequently, there aren't as
many examples of statistically defensible research that these forensics. Forensic

(21:08):
examiners can rely upon that they can go back to
and that they have a depth of research in their field. Right,
And so one argument is we should fund them more
and too they should be broken out into their own division.
They shouldn't be beholden to police and prosecutors because right
now most forensic scientists work under the police and under

(21:29):
prosecutors um, which is obviously going to throw bias into
their work. Yeah, that's another thing that is that is
addressed by that big two thousand nine document is is
experiment or bias? I mean, this is a problem that's
also different in some sense from methodologies. Yeah. Absolutely, Well, actually,
I guess you could say it's part of methodology because

(21:50):
when you design an experiment in science, you part of
your methodology is to try to remove bias, for example,
by blinding or experiment or blinding people carrying out the
research shouldn't be aware. Let's explain what that means. Yeah,
you're not blinding a human being, right, experiment or blinding
for example, is you know, if you want to test
whether a certain type of artificial sweetener causes cancer. When

(22:14):
you're conducting the experiment, ideally the person performing the experiment,
shouldn't know what the hypothesis is, shouldn't know what the
samples they're using are. They should be sort of unlabeled
and identified later by like numbers that could be matched up.
So so that the if the experiment or has a
certain way that they maybe even unconsciously want things to

(22:38):
go in the experiment, that can't affect it because they
don't know what's going on and what's ex there's no
actual variables that they could unconsciously put into the into
the test, right. Yeah, but in many cases in forensic science,
that's not necessarily how it's practiced. A lot of times
the the forensic investigator in certain scenarios might know what

(22:58):
the hypothesis is. It's that this certain person is guilty
and here's what they did. But again that's one that
varies from field to field, so we can't say that
that's problem across the board. Yeah, So, like I'm thinking
like an example here, although I'm not I don't want
to accuse the woman in the case that I was
a part of of of pseudoscience in any way. I
have no idea one way or the other. But her

(23:19):
job was to prove that the cocaine that was given
to her had a certain percentage of pure cocaine in it,
so that this man could be charged for distribution. There's
like on the law, it has to be a particular
percentage so that it's legally enforceable. Um, you know, rather
than it being like, I don't know salt, I don't
know what people mix in with their cocaine nowadays, sorry, guys,

(23:41):
not on the street as much as I used to be. Yeah,
non dairy creamer and salt. The fair powdered here. Yeah, Um,
but she knew what her job was going into it
when she was measuring right. So yeah, you mentioned that
word pseudoscience, and I guess we should get into that
a little bit. Uh. It's something that comes up on
this show fair often we talk about pseudoscience, But what

(24:02):
what is pseudoscience for? For those who are you may
have heard the term, but you're not clear exactly what
it is. You're you're aware that it's maybe just false
information or something. But I think that in this case
it's very important to emphasize the specific definition of it
because pseudoscience US standard Dictionary definition I found is a
collection of beliefs or practices mistakenly regarded as being based

(24:26):
on the scientific method. So um, the scientific method with
reference to forensic science, as defined explicitly in that big
two thousand nine UH National Research Council report is quote
hypothesis generation and testing. So you've got a hypothesis going
in falsifiability and replications, so there are ways that you

(24:47):
could show your knowledge is not true, and testing more
than once by different people to verify the results you get.
And then peer review of scientific publications, so testing your
findings against other experts in your field to see what
they think and if they can criticize your method. Now,
other than the peer reviewed aspect, this is pretty much

(25:09):
what we learned in science class in high school, right, well,
depending on Yeah, when you get your little research lab
book and you're filling out all your stuff for the
experiments that you perform in class and such like that,
this is what they're teaching us. The importance of that
the peer review stuff comes later when you're in I
guess higher education. But uh, it shouldn't be that hard,
you're right, But in a lot of cases that this

(25:31):
is exactly what they have found. Many forensic fields, or
certain forensic investigators don't always practice that their techniques don't
reflect this, or the rules of analysis they use don't
necessarily reflect this. They're not based on well replicated, falsifiable
scientific tests. They're more just kind of like knowledge, you

(25:52):
general wisdom, folk wisdom about here is what you'd expect
to see here, So kind of like, so I like
to think about a scene that I love from TV.
But this is total pseudoscience, all right. But it's also
like street cops right the wire, the infamous f words
seen in the wire, you know what I'm talking about,

(26:13):
Right where bunking Meltier in the kitchen, and they're trying
to figure out how a woman was shot through a
window in the kitchen, and so they're basically measuring with
pens and like I don't even know if they have
a measuring tape in that scene or not, you know
what I'm talking about. And they're measuring like what the
trajectory of the bullet might have been, what angle the

(26:33):
guy might have been shooting from, where she was shot,
where the bullet might have been lodged in the refrigerator door,
all this stuff. In the meantime, while they're swearing a bunch,
which is funny for us as the audience. But that's
not science. Well, I mean, that's just them. They've got
some common sense and yeah, they've they've been on the job,

(26:53):
they've seen enough homicides that they have a bit of
an idea on how to explore that scene. But that's
not something that unst necessarily would submit to the courtroom, right,
I mean, I can I can look at that scene
and say, I don't know, it seems like they're reasonable
in the conclusions they're drawing. Yeah, but um, but yeah,
I mean one problem there is, for example, the lack

(27:14):
of quantitative evidence. Yeah, this is something that's often important
in science, is trying to find a way to represent
your findings with numbers so it takes the gut feeling
out of it. So instead of saying, yeah, that bullet
hole looks like it came from there. Instead what you
should be doing is putting a straight object through it
and measuring with a pro tractor exactly what the angle is,

(27:36):
so that it takes your gut feeling out of it.
You have a specific number that has been explicitly measured. Now,
I think if David Simon, creator of the wire, we're
here with us, he would argue that one of the
points of that scene also is that because the Baltimore
Police Department is so horrifically underfunded, this is how these
police have to go about doing their jobs because they
don't have protractors or forensics teams that can come in

(27:59):
and do all that for them. Right. So, one important
point I think that we should make is that pseudoscience
isn't just any false knowledge or bad epistemology. It's a
specific type of thing. Like if a prosecutor on a case,
you're on a jury, and prosecutor brings in a tarot
card reader, uh to turn some cards to show you
show you exactly how guilty, Uh, you know this guy

(28:21):
is of attacking somebody with a folding chair. That would
be a bad evidential standard. But I don't think I
wouldn't the tower came up, I'd be real worried. Huh.
I wouldn't call it pseudoscience because it's not trying to
dress itself in the wardrobe of science. And that that's
exactly what is so pernicious about some of these flawed
forensic science methods. They take on the credibility of the

(28:44):
scientific method without actually practicing the method in every case. Yeah,
and depending on how charismatic or rhetorically qualified uh, defense
or prosecutor could be. You know, it's very easy to
convince a jury that somebody is an ex it in
something that they just simply don't understand, you know, so
that's also worth considering. Yeah, um, all right, I want

(29:07):
to bring up one thing here that references another old
episode of ours too, which is um, Robert and I
did that episode earlier this year about cargo cult science
and our cargo cultism, and then we also talked about
cargo cult science with a concept from Feineman. H yeah
he was he semi involved in that. Yeah. Yeah. In fact,

(29:28):
we told a really fun story. Go back and listen
the episode. Everybody but told a fun story about Fineman
hanging out in a bathtub while people were doing reflexology
on each other. Uh anyway, Fineman always hanging out in bathtub? Yeah, exactly,
hot tub, I guess more than bathtub there. Um. But
so this also it made me think of what Robert

(29:49):
and I talked about as the Church of science quote
unquote in that episode two, that there is there is
a an idea of science as being almost like a
deity now days, right, that that that true knowledge and
true um ways to judge how the world works can
come from science rather than the old way that we

(30:12):
thought it did, which you know generally was from religion
or mythology. Uh. And this is an example of where
quote unquote science is a false god. Uh. You mean,
like the people who treat science sort of as dogmas
rather than as a method. Exactly when people say, I
hate when I hear this, when people say quote science says, yeah,

(30:35):
that's one of my least favorite phrases. Science doesn't say anything.
Science is a tool. It's not a it's not a
deity speaking pronouncements to you. And one of the people
one of the things scientifically literate people will know is that, um,
you know, scientific results are tentative. You you learn something
through science, but then there might be new studies coming

(30:55):
out next year that say, actually, those previous studies were
flawed and here's the better answer. Yeah, I guess. Just
my general point is that, like, it's important to realize
that this is fallible and that you should apply critical
thinking to this when it's when it's put in front
of you as matter of fact, right, knowledge that is
cloaked in the garments of science ain't necessarily so just

(31:19):
because it's wearing those clothes. Absolutely, So Okay, I want
to bring this back though, because I don't want to
make it seem like we're just totally bashing forensic science
and it's all bad. No were to put your critical
lenses on, and there are examples of well researched science
being used in forensics. In fact, chromatography which was one

(31:41):
of the things we were going to do for the
We're gonna do paper chromatography for our little d I
y uh lab experiment. It's a method for separating complex
mixtures and it allows examiners to identify bodily fluids for
drug cases, and for the most part it's seen as
being well researched and backed up. Uh. The other one
which we mentioned is DNA analysis, which has set a

(32:02):
new scientific standard. But if you also have to remember
that quote, good science takes time, right, so you can't
just like whip up DNA evidence like you know, in
a flat Literally the TV show The Flash, I don't
know if you know this, the character the Flash is
a forensic scientist, and so whenever he's like, you know,
got work to do, he just super quickly like does

(32:23):
it all like in like thirty seconds or whatever. So
it's that's not a good example. Why does he make
the machines work faster? No, he just runs around really
quickly and like shakes like like like little pipettes and
stuff or test tubes really quickly. The machines work faster
because he is the machine. So he becomes a centerfuge
or something like that. Yeah. As much as I love
the Flash, both TV show and just as a character,

(32:47):
it's like the worst example of what should happen in forensics.
SciTE um. Anyways, the DNA thing, it took thirty years
from the discovery of the double helix structure before it
could actually be used to confirm a positive identification of
an individual. Now it's broadly accepted and quantifiable, right, but
it took a long time. In fact, the most advanced

(33:10):
analysis has a one in more than a quadrillion chance
of a random match of two strangers DNA, So that
seems fairly reliable. Yeah. Yeah, Yet DNA only constitutes less
than ten percent of the case load in US crime labs, right,
So that's important to remember as well. What we really

(33:30):
need is the rest of forensics to be just as
rigorous and justice statistically grounded. Yes. Uh. And one thing
I do want to say, picking up on what you
said a minute ago about how we're not trying to
trash all these fields, is that I would say that
I think any field can be pursued in a scientific way.
So if we say something about major flaws in uh,

(33:53):
bite matching or something, or in fire analysis, that's not
to say that forensic ode ontology, this study of you know,
dentistry in in crime cases, or that fire analysis are
not legitimate fields of study. There are totally legitimate ways
to study these, and there are lots of great scientists
who do exactly that. The problem is that as practiced

(34:16):
in many in many court cases and forensic investigations, it's
full of bad methodologies or unverified knowledge. Yeah, and I
think you know that c s I effect that we
were talking about earlier too, is perpetuated by by way
more than just like the sort of um police procedurals
like c s I that we're used to. I mean,

(34:36):
like I just mentioned the Flash that's not really a
cop show, and yet like it throws in a dose
of that kind of stuff in there. Also thinking of
like a great TV show, Sherlock but it relies on
like lots of like examples of Sherlock Holmes conducting his
own like at home forensic tests. Have you ever seen
this show before? Like I've seen was one where like
he's just like beating a corpse with like a horsewhip

(34:57):
or something like that, just just to see, like what
how long it takes bruises to form on a corpse? Right,
And it's like, Okay, like I get it that that's
an interesting like way narratively to show us that he's
conducting research and everything. But also like what but it's
funny that Sherlock is often used as a as a

(35:19):
you know, great symbol of scientific investigation because he uses
deeply unscientific methods a lot of the time. Sherlock Holmes
will you know, look at a few facts about you
and make deductions, right, That's what he does science of deductions.
Say well, your watches turned back to this time, which
makes me know that you were in this country and
this time zone. That's not scientific, that's just making an assumption.

(35:40):
But we love it. We eat it up right, like
as an audience. That's great fun because it makes the
world a lot simpler than it actually is. And it
man like, of course, like I would love it if,
like I could be Sherlock Holmes and just solve every
problem by just like kind of very quickly looking around
the room and knowing everything that's going on. But it's
not realistic. It's a fun story, but in terms of

(36:02):
like people's lives or whether or not they're guilty perpetrators
out there running around, it's not really how things work. Okay,
we need to take a quick break, but when we
come back, we're going to talk about evidence standards for
scientific evidence in US courts and then a bunch of
specific examples of forensic science disciplines and what some problems
with them might be, including, uh, the aforementioned case of phrenology. Okay,

(36:30):
we're back, So Joe tell me and the audience what
is the fry test and how how why is it
pertinent to this forensic research that we're talking about here. Well,
this comes up in a lot of discussions of forensic
science because this has been the legal standard for the
admissibility of scientific evidence in US courts for a long time. Now.

(36:50):
There are other standards that have in some ways superseded it,
but this was sort of the original one. So The
Fry standard essentially determines that for scientific evidence to be
admissible in the court, it has to be quote generally
accepted by experts in the field in which it belongs.
So Fry came out of a murder trial in nineteen

(37:12):
three and which the defendant tried to demonstrate his innocence
by using a lie detector test to measure systolic blood pressure.
So this is an early lie detector test. And and
this guy is going to say, look, you know, I
didn't do it. And here's a machine that shows I'm
telling the truth when I say that I didn't do it. Yeah,
you know what. I think. This is before the polygraph,

(37:34):
because William Marston invented that, and he is also the
co creator of Wonder Woman, the fun things that Christian
Sager knows in his little weird brain, the lasso of truth,
Yeah exactly, And that didn't happen until the late thirties
early forties. That's a whole other field of forensic pseudoscience
that that we can talk about. But so in this case,

(37:55):
the court rejected this evidence, writing quote, just the scientific
principle or discovery crosses the line between the experimental and
demonstrable stages is difficult to define. Somewhere in this twilight zone,
the evidential force of the principle must be recognized. And
while courts will go a long way in admitting expert

(38:16):
testimony deduced from a well recognized scientific principle or discovery,
the thing from which the deduction is made must be
sufficiently established to have gained general acceptance in the particular
field in which it belongs. And that's where this general
acceptance doctrine comes from. It So, lie detector tests not
generally accepted by the relevant scientific fields then or now, really,

(38:39):
and yet we still see them in pop culture all
the time, right, And even if they were mostly accurate,
they're not legitimate scientific evidence to use in a court.
But even this is not fool proof depending on how
it's applied and interpreted. For example, what if a whole
field is just rotten to the core with pseudoscience. Um. So, again,
it might not come as a surprise, but most phrenologists

(39:02):
consider phrenology a legitimate science. Yeah, of course they would.
And so if you don't, here we go, Yeah, here
we go. I see the next line coming up. I'm
gonna I'm waiting for the emails to come in. Right,
most people who own orgon energy accumulators probably think Wilhelm
Reich is not pseudo science. Yeah, absolutely so those of
you who are long time listeners may know. Earlier in

(39:22):
the year, Joe and I did an episode on Willhelm
Reich and organ accumulation and cloud busting, and we received
some nasty emails from people who are deep followers of
Reich's belief. Yeah, they were not happy that we did
not see the genius in his method. Yeah, and I
I like to feel like we did a fair treatment

(39:43):
of that topic, but that they were upset. Yeah, well,
you know, we do our best, but but I don't know.
As we see, I don't think the organ accumulator has
much going on for it. Yeah, and neither do I.
And I'm the one who owns VHS tapes and books
by that guy and has state at his estate. Uh So,
in these cases Wilhelm Reich phrenology, it's obvious to us

(40:05):
what's wrong, because we're well aware of the these fields
and their faults. But what about in obscure sciences, you know,
things that are not popularly known to be uh, pseudoscience,
where an entire field could just be riddled with problems
and nobody on the outside would know UM. So in
nineteen the Federal Rules of Evidence were established, and this

(40:29):
brought in Rule seven oh two which is relevant to this,
and it reads quote, if scientific, technical, or other specialized
knowledge will assist the trier of fact to understand the
evidence or to determine a fact in issue, a witness
qualified as an expert by knowledge, skill, experience, training, or
education may testify there too, in the form of an

(40:50):
opinion or otherwise. So this seems to massively just on
the face of it, this looks like it massively relaxes
the standard um and and this led to much argument,
lots of back and forth between legal scholars like does
this embrace the Fry standard or does it reject and
replace it? And there were many fierce arguments, but eventually

(41:12):
Fry was superseded by the Daubert standard in the courts.
So the Daubert standard is much more complex and it
says um Essentially, the qualifications for the admissibility of scientific
evidence in the courts are whether you can test the theory.
That's an important thing, like it shouldn't be just unfalsifiable

(41:33):
knowledge that there's no way you could show if it
was wrong. Second, whether it has been subjected to peer
review in publication. That's another important one. What it's known
potential error rates are. So you know, you could have
a totally valid field that has a known pretty high
error rate. I mean, it could still be valid as
long as you acknowledge what the known error rate is. Yeah,

(41:55):
and acknowledge that to the jury exactly. Um, the the
existence and maintenance of standards controlling its operation, right, so
you've got to have uh systems in place to to
make sure everybody's doing it right. And uh, and then
again whether it is generally accepted among the scientific community.

(42:16):
So who where did this come from? Yeah? Daubert was
established in nine case in which the plaintiffs were attempting
to show that a drug called ben Deckton had caused
birth defects in their children and uh, and it clarified
this rift between Fry and rule seven O two, saying
that seven O two, as enforced, should have all those

(42:37):
qualities I just mentioned, um, but also that that two
thousand nine National Research Council paper that's like that's like
the big daddy source that we keep coming back to.
If you want to know everything about this. You should
go read that very big. It's learning, it's huge, um,

(42:58):
and that they write that quote. The court also emphasized
that in considering the admissibility of evidence, trial judges should
focus solely on experts, principles, and methodology and not on
the conclusions they generate. And that seems that seems absolutely
crucial and correct to me. What should be important is
the method they use, not what they end up saying. Right, yeah,

(43:20):
and that's still I would say, or at least just
from my experience, not exactly the case, you know, but
the two thousand nine documents says that. I mean, it's
been seven years since then, so hopefully it's better. But
I think that case that I worked on was probably
a couple of years after this. Yeah. But another thing
that this, uh, this indicates is that the court essentially
placed faith in the system by saying, look, we've got

(43:44):
an adversarial system. So you have prosecution and you have defense.
You have plaintiff and you have defense. Since there are
two sides, we can be sort of generous in what
kind of scientific evidence we accept. Because if the defense
doesn't like the prosecut uter's expert witness, they can call
in their own expert witness. Yeah, and so the court wrote, quote,

(44:05):
vigorous cross examination, presentation of contrary evidence, and careful instruction
on the burden of proof are the traditional and appropriate
means of attacking shaky but admissible evidence. That essentially means
it's not the court, or rather, it's not the judge's responsibility. Uh,
it's this. It's on the responsibility of the prosecution and

(44:25):
the defense to argue rhetorically against the science. Yeah. So
the judge might have the ability to rule out a
phrenologist giving evidence, but in a case where that's not
as clear as phrenology, but just where the science appears
legitimate but might be shaky. Uh, it's incumbent upon the lawyers,
right that We'll let it in and we'll see if

(44:46):
the other side has something to say about that. All right, So, phrenology,
you've been you've been teasing around for knology. I know
you've been wanting to tell this story all episode. Okay,
So I'm gonna take you back more than a fifty years.
I'm gonna take you the thirty four the United States,
in the state of Maine. We're up where Wilhelm right, Well,
he wasn't doing an eighteen thirty four but yet right

(45:08):
unconnected early rangely Maine. Yes, this is will not ring
unconnected to Wilhelm Wright. In Maine in eighteen thirty four,
a nine year old boy named Major Mitchell was facing
trial for a serious violent offense. He was charged with
assaulting and maiming another boy, an eight year old schoolmate

(45:28):
named David Crawford, and according to the allegations, Major Mitchell
had lured Crawford into an empty field with the intention
of whipping him and killing him. H Crawford had called
Mitchell names like a hog, a fool, and a steeler,
and then Mitchell began to punch Crawford until an adult
neighbor came by and separated them. Later, Mitchell found Crawford

(45:52):
walking along a road and forced him into some nearby woods. There,
Mitchell began torturing Crawford, he uh supposedly a acording to
the allegations, He filled his mouth with grass, stripped his
clothes off, tied him up, beat him with sticks, tried
to drown him in a stream, including damning the stream
to make the water deeper to make it easier to
drown him, and partially castrated him with a piece of

(46:15):
rusty taka. And this is what happens as you name
your child major. This is messed up. This is not yeah,
not a happy circumstance. That's a joke. By the way,
there's no evidence that naming your child major will scientifically
lead to them being a scumbag. But eventually, but this
is a nine year old kid. This is something weird
going on here. This would be extremely disturbing if an

(46:38):
adult did this to another adult, but this is a kid.
Doing this to another kid is just so messed up
and um So, eventually Mitchell let Crawford go and Mitchell's
legal defense was taken up voluntarily by this Portland's lawyer
named John Neil. Now Neil didn't just do this out
of like pity or kindness. Neil happened to be a

(46:59):
proponent of the budding science quote science and quotes there
of phrenology. So for for the audience out there, and
this is my imagination of phrenology at the time. I
have an inkwell at home that is a bust of
a head and it has all the phrenology. You know
that the stereotypical like phrenology drawings of like which part

(47:20):
of the head does, which it contains which emotions. So
those of you out there who are wondering what does
this phrenology they keep talking about, think of those like
those illustrations of like a profile of a head and
it's like carved up kind of like kind of like
those like depictions of like what parts of a pig
are good to eat? Right? Yeah? Yeah, and it shows
like okay, and this part this is where anger is

(47:43):
and in this part, this is where compassion is. Oh yeah,
it's great. So, uh, it's not great, it's awful, but
it's great. It's great fun to look at it. So
phrenology might in a primitive way be considered a predecessor
to neuroscience, and that it linked mental faculties and personality
traits to the physical makeup of the brain. Uh, who
you are as a product of your brain. That's not

(48:03):
a bad starting place for for science. But from there
it sort of becomes the body builder bro science of
the brain. Uh. So you know how when you lift
heavy weights use certain muscles, those muscles get bigger over time.
I have no idea what you're talking about, at least
in theory, at least in theory. Uh, you know, and
so you can often tell how strong a certain part

(48:25):
of a person's body is by looking at how big
the muscles there are. Well, phrenology sort of takes the
same principle to the brain. It positive that the brain
is covered in these topographical regions. Quote organs, organs right,
that are responsible for particular aspects of personality your behavior.
So if you had a lump or a raised contour

(48:45):
on the part of your head that chronologists legal labeled
the organ of hope or the organ of secretiveness, those
personality traits would be super pronounced in you. So if
one of these guys you know, pulled the let me
tell you to the gun show line, the guns would
probably refer to something like the organ of benevolence, let
me show you my head bumps. So like what what

(49:08):
I mean phrenologists. Even at that time, people knew like
if you get hit in the head with a baseball bat,
bump forms on your head like oh like, depending on
where you got hit in the head, your momentarily turned
into a more benevolent person or a more aggressive person.
That is almost exactly what the defense argued in this case. Okay,

(49:28):
hit it so phrenology is now considered completely discredited. There's
no evidence at all that these cranial organs had any validity. Uh,
And you can't accurately predict people's behavior by measuring bumps
on their heads. But the defense in this case argued
that Mitchell had suffered an injury to the skull when
he was young, quote, whereby the portion of the brain

(49:49):
called by phrenologists the organ of destructiveness was prematurely enlarged
and a destructive disposition excited. Man, if it was that
easy to make children into little monsters, right, like you'd
be able to very easily like create like an army
of homicidal maniacs, but just like whacking them on a

(50:10):
certain part of their head. Well that that's not actually
completely off. And I want to get into that in
a minute. Because so in this case, they observed a
lump behind Mitchell's right here, and they said, look, you know,
the organ of destructiveness is enlarged. Uh. The crime was
a result of this enlarged organ of destructiveness. The enlargement
was due to circumstances beyond Mitchell's control. He just got hurt. Um,

(50:33):
so he shouldn't be held responsible for the crime. And
the judge wrote, quote, where people do not speak from knowledge,
we cannot suffer a mere theory to go as evidence
to a jury. So the judge actually pretty wisely in
this case ruled like this, This like old main judge,
though it sounds like people do not speak from knowledge.

(50:56):
It's like in pet cemetery. Yeah, exactly, you don't want
to go on that road of phrenology. That's a bad
road of phrenology. Okay, yeah, those are a terrible main accent.
I'm sorry. No, mine is not just as equally bad,
and I'm from New England. But this does raise an
interesting question like how to outsiders and lay people determine

(51:18):
what is knowledge and what is mere theory, and how
to judges do it by the preferred nomenclature of today,
I think most scientists would prefer to say mere hypothesis
than mere theory. But either way, um, neither judges determining
admissibility nor jury members listening to an expert witness testify
typically have exhaustive knowledge of these disciplines. So what do

(51:39):
you do? Yeah? But then again, I do think this
case raises important issues that are still relevant today because,
for one, for one thing, courts today will take real
evidence about, say injuries to the brain or illnesses that
affected the brain into consideration when thinking about somebody's responsibility

(52:00):
for a crime. What if you, you know, normal person,
not a violent person, suddenly one day get the urge
to go out and start beating people down with a
folding chair. And then they discover that you've got a
tumor on your brain rick flair disease. The the this
tumor may well be changing your behavior in a way
that it's kind of hard to hold you personally responsible for, right,

(52:23):
I mean, that's that's shaky, but yeah, well, I mean
that's how a lot of the courts would look at it.
And I'm sure you'd feel the same way if you were, Like,
you'd be like, this isn't me. I don't know why
I did that. That doesn't feel like a part of
me at all. And then they discover, well, there's a
tumor in your brain that's changing the way your brain works.
Can you remove the tumor? Well, maybe you can, maybe
you can't. Yeah, I mean I guess like whether or

(52:45):
not you're responsible, you're still responsible. But then then it changes,
like what the punishment would be right, like, like, you
wouldn't necessarily send somebody like that to jail, but they
would need to be ice lated so that they're no
longer doing this if yeah, I mean hopefully they could
get treatment and other things would be a brain injury.

(53:08):
You've probably heard the story of Phineas. I was just
about to bring up Phineas. We talked about him on
the show all the time. Phineas Gage. He's this guy.
What what you're with that? Do you remember? I can't remember,
but I do remember that when I was in a
junior high school that our school had a DARE program
and they would often talk to us about the repercussions
of drinking and driving, and they gave us all t

(53:30):
shirts that had a picture of a skull with a
metal rod shoved up through it, and it was to
remind us about Phineas Gauge and how his brain changed
because of the rod. If you're not familiar with this incident,
he was a railway worker working on a railroad and
and there was some accident and explosion shot a metal

(53:50):
rod through his head and his personality completely. Yea, he lived,
It didn't immediately kill him, but he was not the
same man. And this brain injury had altered something physical
about the way his brain works, and he was no
longer morally or in terms of character, the same person. Right,
And so like in my instance with these T shirts,
Dare was trying to use this as a metaphor for like,

(54:11):
this is what happens when you drink is you're no
longer the same person. Subsequently you shouldn't drive, right, Okay,
But but this is another way that I think neuroscience
may come into forensic science in the future, because like,
the more we understand about neuroscience, the more we understand
about what things of physical changes in the brain can

(54:31):
lead to certain behaviors with with pretty reliable predictive power,
you know, you say, like, oh, we've discovered that most
of the time you see this physical process in the brain,
people start behaving in this way. Uh. Certainly, especially like
when you're talking about depression and anxiety. Yeah, exactly, do
those things start changing our idea of how responsibility works

(54:55):
in the courtroom? Uh? So, Like if we say that
you know, somebody who Phineas Gage had a bar goes
through his head, or somebody had a tumor in their brain,
and this directly seemed to lead to a change in
their behavior that caused criminal activity or something like that. Uh,
and what what if it's not an injury or an
illness but just some inborn condition. This is just how

(55:15):
your brain was born. Yeah. Yeah, it's complicated stuff again,
wicked problems right, Like like it's not it's not even
just as easy as like having the forensic science be perfect,
because even then you get cases like what you're these
hypotheticals that you're putting out. Yeah, it's very complicated to
decide what's right and wrong. Yeah, and since we ran
so long, we're gonna have to call it right there.

(55:37):
So here, here's what we're gonna have to leave off
the first half of this discussion, but to hear the rest,
you can simply tune in again next time. And in
the meantime, you can find us on social media such
as Twitter, Facebook, Instagram, and Tumbler, where our handle is
some variation on blow the Mind. You know what kind
of weirdos we are. You'll be able to tell if
it's our accountant, I'm quite sure. And of course you

(55:58):
can always catch always say and do. It's stuff to
blow your Mind dot com. And if you'd like to
email us to get in touch with ideas for future
episodes or feedback on this one. You can reach us
and blow the mind at how stuff works dot com

(56:20):
for more on this and thousands of other topics. Is
that how stuff works dot com. Remember remember,