July 21, 2016 • 42 mins
What if bad science put you in prison for a crime you didn't commit? How many guilty offenders are still out there because of pseudoscience, fraud or disorganized laboratories? In this episode of Stuff to Blow Your Mind, join Joe and Christian for part two of our review of forensic science in the courtroom. How accurate is the analysis of fingerprints, ballistics, hair, bite marks, fire and DNA?
Learn more about your ad-choices at https://www.iheartpodcastnetwork.com
See omnystudio.com/listener for privacy information.
Mark as Played
Transcript
Episode Transcript
Available transcripts are automatically generated. Complete accuracy is not guaranteed.
Speaker 1 (00:03):
Welcome to Stuff to Blow Your Mind from how Stuffwork
dot Com. Hey, welcome to Stuff to Blow Your Mind.
My name is Joe McCormick, and I'm Christian Sager, and
our regular host Robert Lamb is out on vacation. So
Christian and I are returning to finish our discussion about
(00:24):
science and pseudoscience and criminal investigation and the justice system.
So if you haven't heard part one yet, you should
go back and listen to that first so you can
understand what we're going to be talking about in this episode. So,
without further ado, here is the second half of our conversation.
So Christian, yeah, tell me about fingerprint analysis and ballistics matching. Okay,
(00:46):
So fingerprint analysis is one of those ones that, like,
you know, especially based on the C S I effect,
like a lot of us just assume, like, yeah, well
that's that's accurate, right, Like your your fingerprint is your fingerprint.
Everybody's got a unique snowflake, and uh like how hard
that's got to be bulletproof? Right, No pun intended. There's
(01:06):
a little bit of wiggle room there, right, So legal
experts are concerned. Actually there's inaccuracies in something that's called
the friction ridge analysis that's used in fingerprint identification. So
fingerprints are believed to be unique. The process of matching them, however,
isn't statistically valid, mainly because prints on an ink pad
are compared to smudged or partials, which you always hear
(01:30):
that on shows right, we only got a partial, right
that that's the stuff that's often found at crime scenes.
Fingerprint examiners often testify, however, with absolute certainty. This isn't
like like when you watch I don't know, like a
like a Batman movie or something like that, and he
finds a fingerprint and he runs it through the bat
computer and it's like finds all those points of agreement
(01:51):
and it's a perfect match. But in those examples, it's
usually like a very clear fingerprint matched against a very
clear fingerprint. What if it's a kind of smudge fingerprint
match against a partial, kind of smudged fingerpot exactly, it's
it's much harder to tell, and it's not with absolute certainty.
A two thousand six study by the University of Southampton
(02:11):
in England asked six veteran fingerprint examiners to study prints
taken from their own cases without even telling them where
these came from their results were totally inconsistent. Only two
of the six reached the same conclusion that they had
come to on the second examination. On the first a
(02:32):
pattern recognition expert at Sunny Buffalo is actually developing software
to quantify the certain the certainty of fingerprint matches. So
it's kind of the back computer metaphor that we were
just using. Right, So if you had an automated method, uh,
and it was scientifically valid at the beginning, that would
take sort of the subjectiveness out of it. Well, and
(02:53):
that but also I mean think about it, like it
could also easily tell you how what thecentage of accuracy
was versus rather than like, I don't know, it was
sort of the same, you know, like the well, that's
what I talked about earlier, how it's important to have
numerical quantities to deal with rather than just letting people
(03:14):
go with their gut feeling. Right. So if this program
works the way it's supposed to work, right, it could say, well,
it's got twenty six percent match or it's got a
nine percent match. You know, you just you can't do
that as a human being with your eyes. And that's
essentially what we've been relying on. Right, is like, well
I looked at that one. That I looked at that one,
and I've been doing this for twenty years. They looked
the same to me. So boom, absolute certainty, you know.
(03:36):
And that's a little bit like more complicated than it's
portrayed to be. Ye, same thing happens with ballistics matching again.
Another Yeah, Batman's coming up all over the place in
this one, right, Like was it the Dark Knight where
they were like doing the ballistic right, get that brick
and he finds the bullet and then he matches it
to It's ridiculous. Um. Ballistics matching is often done the
same way, based on the theory that when a bullet
(03:57):
is fired from a gun, it leaves unique marks on
the slug by the guns barrel, but there's no standards
to constitute what operates as a match between bullets. The
National Research Council has actually called ballistics testing into question
and that they say, look, it's neither unique nor is
it reproducible, so why should we be using this in
(04:19):
the court. A lab in Saint Paul, Minnesota, this is
one of the ones I mentioned earlier was found with
major errors that impacted their fingerprint and other evidence processing.
This included sloppy documentation, dirty equipment, as well as a
lack of basic scientific procedure. They actually used Wikipedia as
technical reference. In one case, there was no clean area
(04:43):
designated for Roobe for review even for their DNA evidence. So,
for instance, like we've been saying, you know, we'll talk
about it later at the end of the episode, but
we've been saying DNA evidence is is pretty good, right,
But when you've got like a pig style of a
work environment and there's no clean area to review the
d N a, yeah, it could be tainted. Yeah, And
so these are basic problems with the just like the
(05:05):
environment and procedure of analysis. They might not even be
problems necessarily with the underlying theory that they're using to
uh to determine the outcomes, though there might be problems
there to be, yeah, exactly, So when you combine the
two it gets even worse. Right, So, like ballistics matching
or fingerprint matching, it's not always a d percent certain
(05:25):
there's it's not absolute. And then you throw in the
idea that well, like maybe this fingerprint technician is also
like spilling. I don't know, like a HOGI on his fingerprint.
Uh slides you know everything goes out the window. Yeah,
let us committed this crime, right. Uh So I want
to talk about fire analysis. Yeah, I don't know. It
(05:47):
doesn't want to look at fire. It's it's a glowing
god that draws all of us to it. But so,
getting a joking aside for a minute, this is pretty serious. Actually.
In February two four, a man named you may have
heard about this, A man named Cameron Todd Willingham was
put to death in Texas after being convicted of murdering
(06:08):
his three children by arson. I have not heard of this, Okay.
According to the charges, Williams set fire to his own
house with his children trapped inside in order to kill them.
And Uh, if you've heard about this case before, it
was probably likely from one of the many articles and reports,
maybe the most famous among them being this two thousand
nine article in The New Yorker called Trial by Fire
(06:30):
by David Grant. Uh. And they're all making the case
that the state of Texas had very likely in this case,
executed an innocent man. So why were people saying this? Well,
the main thrust of the cases against the case against
william were the only solid pieces of evidence against him
(06:51):
were Number one, the testimony of a jail house informant
of the criminal informants are sort of notoriously those are
the cis that we were really talking about yeah, unreliable,
yeah uh, and that that's that testimony itself has been
subsequently called into question in this case, and a fire
investigation concluding that the fire showed signs of deliberate arson
(07:12):
pointing to the defendant, how do you come up with that? Well,
I'm gonna get into it. So this fire analysis has
been roundly criticized by experts as being pretty much without
any scientific marriage. So I'm about to quote from David
Grand's New Yorker article in a section where he describes
this scientist and fire investigator named Dr. Gerald Hurst's reaction
(07:33):
to what was happening in the field during fire investigations. Okay, quote.
By the nineties, Hurst had begun devoting significant time to
criminal arson cases, and as he was exposed to the
methods of local and state fire investigators, he was shocked
by what he saw. Many arson investigators, it turned out,
(07:53):
had only a high school education. In most states, in
order to be certified, investigators had to take a four
our course on fire investigation, and pass a written exam.
Often the bulk of an investigator's training came on the job,
learning from Quote old timers in the field who passed
down a body of wisdom about the telltale signs of arson,
(08:15):
even though a study in nineteen seventy seven warned that
there was nothing in the scientific literature to substantiate their validity,
and then later in the piece Grand Rights quote in
nineteen seven, the International Association of Arson Investigators filed a
legal brief arguing that arson sleuths should not be bound
by a nineteen nine Supreme Court decision that's probably referring
(08:37):
to Daubert there Um, requiring experts who testified at trials
to adhere to the scientific method. What arson sleuths did,
the brief claimed, was quote less scientific. By two thousand,
after the courts had rejected such claims, arson investigators increasingly
recognized the scientific method, but there remained a great verys
(09:00):
in the field, with many practitioners still relying on the
unverified techniques that have been used for generations. Quote. People
investigated fire largely with a flat earth approach. Hurst told
me that this means the common sense thing that we
were talking about earlier. Yes, exactly, the quote continues, it
looks like arson, therefore it's arson, he went on. My
(09:21):
view is you have to have a scientific basis otherwise
it's no different than witch hunting. I know arson when
I see it right now. I don't claim to know
one way or another about willing him personally, whether he
was guilty or innocent. But this does make a close
look at the field of fire analysis very worthwhile. Uh And,
and I want to point out that, like pretty much
(09:41):
all of these others in theory, fire analysis is a
perfectly legitimate field of study. It's not for anology. It's
not just wrong from the ground up. So what might
a fire investigation look like? Often it involves searching through
the remains of a burned building for indications of how
a fire was started, what course it took once it
was ignited. Um And in forensic investigation, for fairly obvious reasons,
(10:06):
this often involves looking for indications of arson. You're trying
to figure out, did somebody set this fire or did
it happen accidentally? Uh And if you can find evidence
that accelerants such as gasoline or lighter fluid, where used
to start the fire. That's one of the most common
indications of varson, of deliberate intention in the fire. But unfortunately,
(10:28):
in practice, some fire investigators have been known to use
these highly faulty techniques that are not established to have
any scientific validity. It's just like we're saying earlier, kind
of like folk knowledge. The investigators pass it down, but
there's never been a study dedicated to figuring out whether
these rules of analysis are true or not. So in
(10:50):
a short article in September, the skeptic writer Michael Schermer
reported a conversation he had with a guy named John
Jay Lntini, who is a fire analysis expert and the
author of this this criminal this forensics textbook called Scientific
Protocols for Fire Investigation. So this is a guy who's
taking the scientific approach, and Lean Teeny tells him that
(11:11):
lots of fire investigators follow scientifically baseless folk wisdom, such
as that so called alligator ing can indicate whether a
fire burned fast or slow. And this alligator ing, this
this idea is that it's it's like looking at an
alligator's hide. So you look at wood that burned during
a fire, and if you look at it and you
(11:31):
see small flat blisters, that means that the fire burned
quote slow. And if you see large shiny blisters, that
means that it burned quote fast. Lntini his analysis of
this rule is quote nonsense. Yeah. Ok, that nobody's actually
measured that essentially, and then we quantified it and published
(11:53):
papers that we can refer back to. Right. Also, according
to len Tina, it used to be fire investing gator
wisdom that when you see crazing and window glass, you
know what crazing is. It's like when you see these, uh,
these sort of crazy spiderweb patterns virgin not exactly, not
like the circular spiderwebs, but just cracks all through the glass,
crazy all over the place. Right, This glass is now
(12:19):
legally insane. Yeah, So they uh, they used to say
that this was an indicator that the glass had been
heated very rapidly, which would indicate the use of an accelerant.
It heated rapidly because somebody squirted gasoline or lighter fluid
all over the place. In fact, that's not true, and
it turns out after scientific investigation that window crazing is
(12:40):
caused by the opposite. It's caused by rapid cooling, like
what would happen when firefighters douse the area with water. Okay,
I can picture this, yeah yeah. Um. Also another thing
that's common puddle shaped burns on the floor that can
make it look certainly like there was an ignited pool
of liquid on the floor, like somebody emptied a gas
can all over the place set it on fire. According
(13:02):
to Lenten E wrong again, even though fire and heat
do tend rise when an entire room burns, the floor
burns along with it, and these marks can appear without
the presence of an accelerant. Um. So, fire analysis is
one of the forensic fields addressed in this two thousand
nine in our C report, and they conclude that while
there is a fairly solid basis for explosion analysis like
(13:25):
when a bomb goes off, fire analysis is much shakier
than explosion analysis is as practice today. Um. They say,
quote many of the rules of thumb that are typically
assumed to indicate that an accelerant was used e g.
Alligatoring of wood specific char patterns have been shown not
to be true. Experiments should be designed to put arson
(13:46):
investigations on a more solid scientific footing. Uh. And again
this is just crazy. I mean, this is yet another
one of these that has been used to convict people,
like in the case of William So we don't know,
I guess whether he was guilty or innocent, But if
this was the main evidence to show he was guilty,
that's that makes it look pretty bad. Yeah, that's kind
(14:07):
of stuff is depressing, especially like when you I don't
know if you watched Making a Murderer, right, when you
watch these like long form documentaries about the justice system
and just how flawed it is. Not saying that in
that particular case, I know whether anybody is guilty or
innocent or anything, but it's you think about how often
this stuff goes on and then you've got examples like this.
It's heartbreaking. Uh. This section though, highlights one thing I
(14:30):
do want to emphasize again where we don't want to
paint all forensic science professionals and experts in these fields
with a with a you know, tainted brush, I mean,
and also saying here like most of them are doing
the best they can with what they have available to
him exactly, and in this case, the bad forensic science
and fire investigation is being exposed by good scientists and
fire investigation so it's not like everyone in this field
(14:52):
is doing a bad job of it, but many people
who are practicing it are. Yeah, this is truly a
wicked problem. I wish we had more money to divide
to divert into research and resources for stuff like this. Yeah. Well,
hair analysis is one of the ones we brought up
at the top because that was actually one of the
hair and fiber was one of the things we're talking about,
(15:13):
Josh about. Yeah, we're and we were also like, maybe
we'll do something like that in our little like video series,
and then the more we start to look into it,
we're like, I don't know about this, this is a
little hokey. Well okay, so one thing I can say is,
if you find a perpetrator's hair at the crime, couldn't
you use that to do some DNA analysis? That would
be pretty conclusive. I don't know, I would think so.
(15:35):
But again, remember the statistics on how often they use
DNA and say versus something like this, probably because of
the timing and the money involved. Yeah, um, whereas like
to just look at two fibers and have somebody come
in as an expert and say, yep, it's a match.
Seems easier to a lot of people, I would guess
are cheaper. Okay, what happens when there is hair matching
(15:58):
or hair fiber analysis. Well, I want to back up,
because there was just last year the Justice Department and
the FBI formally acknowledged that nearly every examiner in the
FBI's laboratory on microscopic hair comparison gave flawed testimony in
the trials that they offered evidence. And this is over
(16:20):
the course of two decades. This is huge. It's like
the biggest UH scandal in forensic science to date as
far as I can tell. Um. They were giving statements
that were quote, beyond the bounds of proper science. What
they were basically doing was saying there's a hundred percent
match between two hair fibers or a hair fiber found
(16:42):
on the scene the hair from you know, a defendant, uh,
when there's just a similarity. So they're totally misleading. The
jury's twenty six out of the twenty eight overstated forensic
mass matches favor the prosecutors, of course, because they're working together.
This equated to nine of the two hundred sixty trials.
(17:02):
According to the National Association of Criminal Defense lawyers and
the Innocence Project. This is the nation's largest post conviction
review of question forensic evidence. It's crazy. Of these cases,
thirty two defendants were sentenced to death. Fourteen have already
been executed or died in prison. So that's I mean,
(17:26):
whether they were innocent or guilty, we don't know, but
they don't yeah, and we'll never have a chance. Now
the cases against them have been undermined. And keep in
mind that just because the FBI made these errors, there
was other evidence used to convict these defendants of guilt. Okay,
so it wasn't the only thing. But still each case
(17:47):
has to be reviewed to see if there are grounds
for an appeal. Four were previously exonerated, So this is
a landmark revelation that pattern based forensic techniques, things like
hair bite mark analysis, ballistics largely subjective, and these contributed
to more than twenty five of the three hundred and
(18:08):
twenty nine cases where a defendant has been exonerated with
DNA analysis post conviction. So that's how DNA analysis has
been used very much in the last like ten to
twenty years. Well, we talked about the exclusionary principle earlier,
you know, a lot of what DNA evidence is actually
brought to the field is uh, not just matching and
saying here, you know this criminal, definitely, that's that's his
(18:29):
or her blood there, but excluding people saying, look, this
is not their day genetic material. Yeah, this is not
a match. So a lot of politicians weighed in on this.
Of course, they asked the FBI to do a systematic
analysis to breakdown their system, and the FBI says that
their hair examiners lacked written standards that defined what were
the appropriate ways to explain their results in court. That
(18:52):
was until two thousand and twelve. This all came to
light after the Washington Post reported on flawed forensic hair
match is. Federal authorities investigated this and they found it
to be true. They found that the experts quote unquote
that we're testifying to near certainty of matches in a
crime scene, they were citing incomplete and misleading statistics drawn
(19:14):
from their own work. So I want to make this clear,
there is no accepted research on how hair from different
people can appear the same. So, I mean, like you
asked at the topic, well, how does that work? I
mean I think it's literally like you look under a
microscope at the two pieces of hair, and that looks
similar to me. Yeah, And I mean I'm sure that
there's some more to it than that, right, in the
(19:36):
same way as like there's some old timey wisdom passed
down about how fires look the same, right, but there's
no accepted research that one one thing leads to another.
Even before this, the FBI reported it's examiners were reporting
false hair matches more than eleven percent of the time,
and at the time of this report, five defendants were
(19:58):
exonerated through d in a testing, all of whom had
served between twenty and thirty years in prison for either
rape or murder. Jeez, I mean, this is just, again,
like I said, heartbreaking. Of twenty one thousand federal or
state requests for hair comparison evidence from nineteen seventy two
to nineteen ninety nine, the FBI found that two thousand,
(20:20):
five hundred of those cases had examiners declare hair matches.
They're reviewing every single one of those cases now, like
they have to go back and just review everything. The
same examiners who are now under review also taught five
hundred to a thousand state and local crime analysis labs
(20:41):
how to testify in the same way. So this is
like the old timers passing down the wisdom, and so
it's an endemic problem. It's not just the FBI, it's
everywhere now. So these same testimonies were likely flawed at
the same level. It's just it's it's insane. In one
shocking example, this is just one example of how it
affected a person, a lot person's life. Sante Tribble served
(21:04):
twenty eight years for murder based on FBI hair testimony.
It later turned out that one of the hairs that
was used to prosecute him came from a dog. Oh
my god, he was exonerated in Wow, So hair analysis
not so much bite mark analysis. Okay, here's another example.
(21:28):
Expert in forensic odeontology, we mentioned that earlier testified that
multiple bite marks found on a murder victim were entirely
consistent with the dental impressions taken from a guy named
Roy Brown. This was the only physical evidence in this case.
Brown was sentenced to twenty five years to life in prison.
DNA tests later confirmed that a second suspect was actually guilty,
(21:50):
but Brown. He spent fifteen years in prison. So bite
mark analysis is now also widely considered unreliable. Yeah, it
was explored in the two is a nine in RC
document and that they outlined several problems with current use
of bite mark evidence. Uh, they say, quote, uniqueness of
human dentition has not been scientifically established. I mean you'd
(22:12):
think that that's pretty basic. You'd have to start with
the study making it statistically clear that humans have unique
bite marks. But I mean, what if they don't usually
have unique bite marks. I mean, I don't know what
you would expect bite marks to look like. Maybe lots
of people have very similar looking bite marks. And I
would also think that again, like this is I haven't
(22:34):
done research on this, but I would imagine that the
force of the bite would contribute to what the bite
marks look like. Well, that's another thing. So they say,
even if dentition is unique, even if everybody has a
unique bite mark in the same way that it's commonly
assumed people have unique fingerprints. Uh, the ability of the
(22:55):
person to transfer a unique pattern to human skin, and
then the ability of the skin to maintain the uniqueness
of that pattern has not been scientifically established. So, so
imagine just what happens when people are trying to analyze
a bite mark on skin that maybe sagging or distorting
the bites. The marks might not leave a clear impression
(23:17):
that's still there when the forensic scientist gets a picture
of it or gets to examine it. I don't even
know how you test this, like, well, I meanlock style,
Like you get like some molds of teeth and then
you take like a corpse and and have the molds
bite the corpse several times and see how they line up. Yeah,
I think you'd have to. You'd have to examine it
(23:37):
statistically with a large sample size, just testing different Yeah,
different molds of teeth or something like that. I don't know,
if you test on dead people's bodies or something like that. Well,
I don't know. I mean, it's the thing. I admit
this is difficult to test, but this is this is
ground level research that should be done if you're going
to be using this to convict people. Oh yeah, yeah,
yeah absolutely, or to defend people either way. All right,
(24:01):
so you're ready for the big one. Yeah, So DNA
analysis we established at the beginning, and I think this
is true, is widely considered, and I think with good reason,
the most reliable forensic science field. When you when you're
matching DNA evidence, you can know if the if the
procedures have been carried out carefully and and all of
the protocols have been followed, that if you have a
(24:23):
DNA sample of the crime scene, and you have a
DNA sample from the defendant, you can know, you know,
beyond a one in billions chance of error, that this
is the same person or it's not the same person. Yeah.
Except The Atlantic published an article this year called a
reasonable doubt. Uh, and it said exactly that it starts
(24:46):
off and saying yes. D NA analysis has long been
held as the exception to the rule about forensics. Arose
from academics. It's been studied and validated by researchers all
around the world. Uh, just a little history here. It
was pioneered by British geneticist named Alec Jeffreys. He was
looking into genetic sequencing and he applied it in the
(25:06):
field with police to help solve a pair of murders
in the British Midlands. Following that, several private companies in
the US and the UK opened their own forensic DNA labs.
But defense teams have argued against it for years. Well why, Well,
the first thing they said was, at first DNA analysis
actually didn't pass the fry test. I can understand when
(25:28):
it's new, it's not generally accepted yet. Yeah, they set
a legal standard requiring scientific evidence to have widespread acceptance
in his field was needed. Okay, well we have that now.
Academics complained that these firms weren't actually being transparent about
their testing technique. Well, our methodology wasn't. Well, that's a
perfectly acceptable concern. I think anybody doing this kind of
(25:49):
analysis should be making public exactly what their methodologies are.
And then, as we've well, I maybe not all of us.
I I was paying attention to the news at the time,
but as popularized by the O. J. Simpson trial, the
argument that DNA samples can in fact be contaminated during
both collection or in the crime lab. But throughout the years,
(26:12):
the thoroughness of labs and analysis list they all got better, right,
so so much so that the Innocence Project, which we've
been mentioning over and over again this episode, was founded
in because they were absolutely convinced that DNA evidence could
exonerate questionable convictions. They have since won one hundred and
seventy eight exonerations due to DNA testing. In a book
(26:35):
by the founders of the Innocence Project, they said, quote,
DNA testing is to justice what the telescope is for
the stars. But yeah, I think in many ways that's true.
And now we're gonna say what the flaws are. But overall,
DNA testing, I will agree, is super solid. Yeah, yeah, no,
I think you're right too. But it's important to sort
(26:56):
of lay this out to know that there are problems
with it. Uh So this has been amplified, of course
by the c s I effect that we've also been
talking about, right the expectation of jurors to see DNA
evidence in corn cases. In fact, in two thousand and eight,
there was a study done by a felony judge in
Michigan where he randomly pulled a thousand, twenty seven summon jurors,
(27:20):
and seventy five percent of them expected that they would
see DNA in a rape case. Fifty of them expected
that in a murder or an attempted murder case. They
would see DNA in expected they would see it in
any criminal case that they were a part of. The
DNA was just so widely available to research and use
as evidence that of course it would pop up. Yeah,
(27:41):
and this has got to be frustrating to I mean,
well meaning prosecutors. Who are you know, they're they're not
trying to cut corners or something, But in many cases
DNA evidence just might not be available. Well, and the
other side of it, too, is that DNA has become
such a powerful tool in the courtroom that it almost
automatically secures convictions, right because people hear those words d
(28:03):
N A and they're just like, well, that's irrefutable. Well,
I mean, I can say if if there's a murder
case and uh, and there are blood stains found on
a victim, and you can take those blood stains and
and several different DNA analysis labs, I'll say that it's
a perfect match for the defendant. I think it's pretty
likely that looks like guilt to me. You should watch
(28:25):
Making a Murderer Man, because there's a whole thing in
that about how like they found this guy's blood at
a at a crime scene in a car. And then
the argument is made, well, the police actually had access
to his blood from previous case. They may have planted
the blood there. Well, in that case, that's I would
say that's again not a problem with the scientific methodology,
but a problem with the with the investigation and the procedures. Yeah,
(28:49):
like you're right, so okay. Research from Australia found that
sexual assault cases involving DNA evidence were twice as likely
to reach trial and thirty three times is likely to
result in a guilty verdict. There's DNA right. Homicide cases
fourteen times is likely to reach trial, twenty three times
is likely to end in a guilty verdict. Another major
(29:11):
study in the UK found that just the knowledge that
prosecution intended to introduce DNA evidence before it was even
actually like introduced in a case was enough to get
a defendant to capitulate because they're just like, okay, so
there's questions about collection and storage, but they just they
just stopped because what do you mean stopped? Well, because
(29:32):
people were starting to think, like, well, this is just
irrefutable evidence. Why should we bother looking at the collection
and storage of it. DNA is just winning so many cases,
let's not bother with it. So as the cases we
use DNA evidence and become more complicated, it actually becomes
less of an objective science. And I'll give you an
example here so just real quick primer on this, nine
(29:55):
percent of our genes are the same as every other
human on the planet. My genes are the same as
Joe's jeans. But the DNA analysts know this, of course,
So we need to look for things that are called alleles.
They're very specific locations on each DNA strand that vary
from individual to individual, the different versions of a gene
(30:15):
in the different places on your chromosomes. The standard is
to compare alleles at thirteen locations. Now, if you do
it that way, the odds of two unrelated people matching
are less than one in a billion, So that's pretty good, right,
But what happens if you're looking at a case and
it involves a third person, So you're not just comparing
(30:36):
two DNA, you're comparing three, or what if you're comparing four?
What if there are four people at the scene. Figuring
out which alleles belong to whom becomes more complicated the
more people that are involved, And if a sample is
small or degraded, which they often are from crime scenes,
the alleles might drop out in some locations as well.
So there's a study done by a guy's name a
(30:58):
teal drawer and Greg hamp A key in they took
DNA paperwork from a two thousand to Georgia rape trial
here in the state that we're recording this in. They
gave the evidence to seventeen technicians without context. All of
these people were experienced DNA technicians with an average of
nine years in the field. They were asked to determine
(31:20):
if the DNA was from the defendant in the case.
Only one of the seventeen people concurred that it was
the defendant, twelve said it was exclusionary, for said it
was totally inconclusive. Since this example, there have been accounts
of dozens of DNA typing cases that have gone wrong,
especially because it's so easily contaminated depending on who comes
(31:43):
in contact with it. Right. So there's something that's called
d N A transfer which contributes to this the most.
It's when cells migrate from people to people or from
people to objects, inevitably when we just touched things. Right.
Another study asked participants to shake hands for two minutes
and hold a knife when the DNA on the knife
was analyzed. Well, to be a participant there just sitting
(32:08):
there holding the knife, and now you hold the knife.
What do I do with it? Just hold it? Just
hold it. Man. When the DNA on the knife was analyzed,
the partner was identified as a contributor in the cases
found them to be the sole contributor. They didn't even
find the DNA from the second person. So going forward,
more context needs to be around for this DNA analysis. Right.
(32:32):
The science is is there, but it it's complicated and
there needs to be like, I guess again, procedures put
in place for like, well, how how do you what?
What's the percentage of accuracy? I think is what we
need to come down on. Uh. Houston, Oh boy, Houston.
(32:53):
Houston had a real bad incident of this. Their crime
lab handled DNA evidence from five cases a year. A
local television station obtained dozens of the DNA profiles that
were processed by this lab and they sent them to
independent experts for analysis. It turned out that the technicians
were routinely misinterpreting even the most basic of DNA samples.
(33:15):
So this was just like a technician error, and the
Innocence Project themselves said, well, we don't take cases where
there are positive DNA matches. And one example, a tech
created a profile for a victim from three different sets
of DNA, the result profiles all totally varied. Then she
mismatched the DNA from the crime scene and the accused defendants.
(33:38):
This led to a retrial where the defendant was released.
So this tech was fired. This is the same Houston
lab she was fired. Then she was reinstated because her
lawyer said, well, the problem was systemic. It was the
whole lab in Houston that was the problem here. They
had inadequate supervision. Well that may have been the case,
it could have been. Yeah, but think about all five
(34:00):
of those cases. I mean, we think that this DNA
analysis is rock solid, and then you hear this and
it's like, well, there's five cases we have to go
back and review again. You know. So, oh man, this
is so crazy because it's you think of scientific evidence
as being the most reliable kind of evidence that you
(34:21):
can have in a courtroom, at least I would, I think.
But and and that's so scary that you often will
have this problem of not knowing whether you can trust
the expert witness telling you something because you don't know
if there if there was something wrong with the data
collection and contamination. You don't know if their methodologies are
actually sound to begin with, whether they've been based on
(34:45):
well tested scientific principles, do they have a clean lab.
Is their supervisor pushing them to do more work than
they're capable of doing exactly? Or is there is there
prosecutorial bias or something. Is there pressure from the dep
artment for them to uh basically convict more cases? Yeah?
(35:06):
And man, that's so tough because we've already seen evidence
in courtrooms undermined by lots of other kinds of studies.
I mean, the more we learn about the reliability of
eyewitness testimony, nothing ever makes it look more reliable. Uh. People,
it just turns out, man, people's memories are not very good.
(35:26):
And in many cases, people have they profess confidence in
identifying a you know, a perpetrator or something in a courtroom,
but then it turns out there wrong. And uh so,
so eyewitness testimony. What I'm not saying it's always wrong,
but it's just you can't be very confident in trusting it. Um.
(35:47):
We we've seen all these problems with forensic science and
so man, it is just it's just tough out there
to know that you're getting the right answer in a courtroom.
I would hate to be I would hate to be
the defendant in any case, especially a case in which
I was innocent. Right now, that's yecially if you're innocent, Yeah,
I think, yeah, it's guilty. I'd still hate it, but
(36:09):
I'd be like, but I did beat Joe down with
the folding chair, Yeah exactly, But how are they going
to tell bruise marks? There's no science that unless you're
Sherlock and you're beating corpses with a horse whip, like
I said, Uh, okay, jokes aside, Seriously, this is a
big problem. What do we do, like, what's the future
of this look like? How do we reform it? Well?
(36:31):
One of the problems is that forensic science encompasses many
different fields, right, so that lots all these fields have
different standards, and some have different problems. Some are more
solid than others, Some have more regulations in place to
keep all of the practitioners in line than others do. So,
so it's a it's a big, hairy, complicated problem, and
(36:53):
there's not one simple prescription that will fix it all. Yeah,
but there are some sort of general rules that some
fields may already be following pretty well, and others could
do a much better job of Yeah, some possibilities that
people are throwing out as they're saying, defendants should have
their own forensics experts that are paid for. Uh. There
should be separate crime labs from the prosecutors and the
(37:14):
police that they're answering to. And there should be, of course,
as we've been alluding to all episode, an established system
of verification and standards. Yeah. I mean one one hypothetical
you could imagine would be something like that. There is.
It would be sort of like the uh, forensic science
version of public financing of elections, where you say, you know,
(37:38):
like if you have mismatched resources going in, you can
make this fairer if you just give both sides equal
money from the government and that's what they have to
spend on elections. Uh. In in this case, you could say, well,
what if we just have a a crime lab that is,
you know, big national crime lab that works for neither
the defense nor the prosecution, and that in order to
(38:01):
get forensic science that's admissible in court, you have to
go through this this big uh, this big well regulated system,
and that that they don't work for either side. Yeah.
In fact, an independent entity that would be called the
National Institute of forensic science has been recommended for something
like this, and they would be responsible for both establishing
(38:23):
the standards that we're talking about and for certifying people
as experts. Another another way of helping the problem is
just from the ground up in each field itself. Like
we were talking about earlier in in fire analysis, they're
very legitimate scientists working in this field who are just
trying to improve the field by doing scientific research and
holding others in their field accountable to pay attention to it. Yeah, yeah,
(38:47):
that's true. Uh. There's also this one like case that
came up about DNA, which is interesting. There's a company
that's using software and automation to take the human element
out of it completely, and I've alluded to that as
well with like finger analysis things like that. Um, they're saying, well,
we take the subjective decisions out of it, and the
analyst is the one who's making the flawed conclusions. The
(39:09):
company calls itself true Allele uh. And the software, however,
has been criticized for whether it actually met the FRIES standard,
even though we're no longer using the price standard is
being criticized because it hasn't been accepted by the larger
science community. But one thing I would say about that
is I think if you have a computer program that's
that's doing analysis on forensic science, I think that would
(39:30):
need to be open source. And company who develops that
probably isn't gonna want that because they're gonna want it
to be proprietary own it. But that should be something
that anybody can go look at the source code and
make sure that it's working accurately and not biasing towards
a certain conclusion. Okay, so that was a lot about
(39:53):
forensic science, and unfortunately we don't this is already a
long episode. We don't have time to two more here,
and there were a bunch of other topics we wanted
to get into and just didn't even have space for,
like handwriting analysis, lie detection, the polygraph recovered memories. So
maybe we can address those. If you're interested, you can
write us and let us know you want to learn
(40:13):
about them, and we'll talk about them in a future episode. Yeah,
we can sit down record a future episode, maybe bring
in old Robert Lamb and uh we could uh um. Also,
like I would love to hear in the meantime from
some of you who perhaps work in this field, you know,
we have something to add to the discussion. Yes, is
there anything we missed out on or that's a really
interesting case of of of how these standards are enforced
(40:37):
in the field that you're familiar with. So if you
want to write into us about forensic science, you want
to talk to us about maybe your experiences, maybe you've
been on a jury like I have, or you work
in forensic science lab and you have some argument perhaps
with how we've portrayed the science here. Please let us know.
We'd love to hear from you. We are all over
(40:58):
the internet. You can find us on social media on Facebook, Twitter, Tumbler,
and Instagram now right. You can also visit us at
our home base, stuff to Blow your Mind dot com.
That's where you can find all of the podcasts we've done.
We also right there and all of our videos are
there as well. And of course, if you want to
email us and let us know your feedback on this
(41:19):
episode or any other, or just let us know a
topic you might like us to do in the future,
you can email us at blow the Mind at how
stuff works dot com for more on this and thousands
of other topics. Is that how stuff works dot com
Welcome to Stuff to Blow Your Mind from how Stuffwork
dot Com. Hey, welcome to Stuff to Blow Your Mind.
My name is Joe McCormick, and I'm Christian Sager, and
our regular host Robert Lamb is out on vacation. So
Christian and I are returning to finish our discussion about
(00:24):
science and pseudoscience and criminal investigation and the justice system.
So if you haven't heard part one yet, you should
go back and listen to that first so you can
understand what we're going to be talking about in this episode. So,
without further ado, here is the second half of our conversation.
So Christian, yeah, tell me about fingerprint analysis and ballistics matching. Okay,
(00:46):
So fingerprint analysis is one of those ones that, like,
you know, especially based on the C S I effect,
like a lot of us just assume, like, yeah, well
that's that's accurate, right, Like your your fingerprint is your fingerprint.
Everybody's got a unique snowflake, and uh like how hard
that's got to be bulletproof? Right, No pun intended. There's
(01:06):
a little bit of wiggle room there, right, So legal
experts are concerned. Actually there's inaccuracies in something that's called
the friction ridge analysis that's used in fingerprint identification. So
fingerprints are believed to be unique. The process of matching them, however,
isn't statistically valid, mainly because prints on an ink pad
are compared to smudged or partials, which you always hear
(01:30):
that on shows right, we only got a partial, right
that that's the stuff that's often found at crime scenes.
Fingerprint examiners often testify, however, with absolute certainty. This isn't
like like when you watch I don't know, like a
like a Batman movie or something like that, and he
finds a fingerprint and he runs it through the bat
computer and it's like finds all those points of agreement
(01:51):
and it's a perfect match. But in those examples, it's
usually like a very clear fingerprint matched against a very
clear fingerprint. What if it's a kind of smudge fingerprint
match against a partial, kind of smudged fingerpot exactly, it's
it's much harder to tell, and it's not with absolute certainty.
A two thousand six study by the University of Southampton
(02:11):
in England asked six veteran fingerprint examiners to study prints
taken from their own cases without even telling them where
these came from their results were totally inconsistent. Only two
of the six reached the same conclusion that they had
come to on the second examination. On the first a
(02:32):
pattern recognition expert at Sunny Buffalo is actually developing software
to quantify the certain the certainty of fingerprint matches. So
it's kind of the back computer metaphor that we were
just using. Right, So if you had an automated method, uh,
and it was scientifically valid at the beginning, that would
take sort of the subjectiveness out of it. Well, and
(02:53):
that but also I mean think about it, like it
could also easily tell you how what thecentage of accuracy
was versus rather than like, I don't know, it was
sort of the same, you know, like the well, that's
what I talked about earlier, how it's important to have
numerical quantities to deal with rather than just letting people
(03:14):
go with their gut feeling. Right. So if this program
works the way it's supposed to work, right, it could say, well,
it's got twenty six percent match or it's got a
nine percent match. You know, you just you can't do
that as a human being with your eyes. And that's
essentially what we've been relying on. Right, is like, well
I looked at that one. That I looked at that one,
and I've been doing this for twenty years. They looked
the same to me. So boom, absolute certainty, you know.
(03:36):
And that's a little bit like more complicated than it's
portrayed to be. Ye, same thing happens with ballistics matching again.
Another Yeah, Batman's coming up all over the place in
this one, right, Like was it the Dark Knight where
they were like doing the ballistic right, get that brick
and he finds the bullet and then he matches it
to It's ridiculous. Um. Ballistics matching is often done the
same way, based on the theory that when a bullet
(03:57):
is fired from a gun, it leaves unique marks on
the slug by the guns barrel, but there's no standards
to constitute what operates as a match between bullets. The
National Research Council has actually called ballistics testing into question
and that they say, look, it's neither unique nor is
it reproducible, so why should we be using this in
(04:19):
the court. A lab in Saint Paul, Minnesota, this is
one of the ones I mentioned earlier was found with
major errors that impacted their fingerprint and other evidence processing.
This included sloppy documentation, dirty equipment, as well as a
lack of basic scientific procedure. They actually used Wikipedia as
technical reference. In one case, there was no clean area
(04:43):
designated for Roobe for review even for their DNA evidence. So,
for instance, like we've been saying, you know, we'll talk
about it later at the end of the episode, but
we've been saying DNA evidence is is pretty good, right,
But when you've got like a pig style of a
work environment and there's no clean area to review the
d N a, yeah, it could be tainted. Yeah, And
so these are basic problems with the just like the
(05:05):
environment and procedure of analysis. They might not even be
problems necessarily with the underlying theory that they're using to
uh to determine the outcomes, though there might be problems
there to be, yeah, exactly, So when you combine the
two it gets even worse. Right, So, like ballistics matching
or fingerprint matching, it's not always a d percent certain
(05:25):
there's it's not absolute. And then you throw in the
idea that well, like maybe this fingerprint technician is also
like spilling. I don't know, like a HOGI on his fingerprint.
Uh slides you know everything goes out the window. Yeah,
let us committed this crime, right. Uh So I want
to talk about fire analysis. Yeah, I don't know. It
(05:47):
doesn't want to look at fire. It's it's a glowing
god that draws all of us to it. But so,
getting a joking aside for a minute, this is pretty serious. Actually.
In February two four, a man named you may have
heard about this, A man named Cameron Todd Willingham was
put to death in Texas after being convicted of murdering
(06:08):
his three children by arson. I have not heard of this, Okay.
According to the charges, Williams set fire to his own
house with his children trapped inside in order to kill them.
And Uh, if you've heard about this case before, it
was probably likely from one of the many articles and reports,
maybe the most famous among them being this two thousand
nine article in The New Yorker called Trial by Fire
(06:30):
by David Grant. Uh. And they're all making the case
that the state of Texas had very likely in this case,
executed an innocent man. So why were people saying this? Well,
the main thrust of the cases against the case against
william were the only solid pieces of evidence against him
(06:51):
were Number one, the testimony of a jail house informant
of the criminal informants are sort of notoriously those are
the cis that we were really talking about yeah, unreliable,
yeah uh, and that that's that testimony itself has been
subsequently called into question in this case, and a fire
investigation concluding that the fire showed signs of deliberate arson
(07:12):
pointing to the defendant, how do you come up with that? Well,
I'm gonna get into it. So this fire analysis has
been roundly criticized by experts as being pretty much without
any scientific marriage. So I'm about to quote from David
Grand's New Yorker article in a section where he describes
this scientist and fire investigator named Dr. Gerald Hurst's reaction
(07:33):
to what was happening in the field during fire investigations. Okay, quote.
By the nineties, Hurst had begun devoting significant time to
criminal arson cases, and as he was exposed to the
methods of local and state fire investigators, he was shocked
by what he saw. Many arson investigators, it turned out,
(07:53):
had only a high school education. In most states, in
order to be certified, investigators had to take a four
our course on fire investigation, and pass a written exam.
Often the bulk of an investigator's training came on the job,
learning from Quote old timers in the field who passed
down a body of wisdom about the telltale signs of arson,
(08:15):
even though a study in nineteen seventy seven warned that
there was nothing in the scientific literature to substantiate their validity,
and then later in the piece Grand Rights quote in
nineteen seven, the International Association of Arson Investigators filed a
legal brief arguing that arson sleuths should not be bound
by a nineteen nine Supreme Court decision that's probably referring
(08:37):
to Daubert there Um, requiring experts who testified at trials
to adhere to the scientific method. What arson sleuths did,
the brief claimed, was quote less scientific. By two thousand,
after the courts had rejected such claims, arson investigators increasingly
recognized the scientific method, but there remained a great verys
(09:00):
in the field, with many practitioners still relying on the
unverified techniques that have been used for generations. Quote. People
investigated fire largely with a flat earth approach. Hurst told
me that this means the common sense thing that we
were talking about earlier. Yes, exactly, the quote continues, it
looks like arson, therefore it's arson, he went on. My
(09:21):
view is you have to have a scientific basis otherwise
it's no different than witch hunting. I know arson when
I see it right now. I don't claim to know
one way or another about willing him personally, whether he
was guilty or innocent. But this does make a close
look at the field of fire analysis very worthwhile. Uh And,
and I want to point out that, like pretty much
(09:41):
all of these others in theory, fire analysis is a
perfectly legitimate field of study. It's not for anology. It's
not just wrong from the ground up. So what might
a fire investigation look like? Often it involves searching through
the remains of a burned building for indications of how
a fire was started, what course it took once it
was ignited. Um And in forensic investigation, for fairly obvious reasons,
(10:06):
this often involves looking for indications of arson. You're trying
to figure out, did somebody set this fire or did
it happen accidentally? Uh And if you can find evidence
that accelerants such as gasoline or lighter fluid, where used
to start the fire. That's one of the most common
indications of varson, of deliberate intention in the fire. But unfortunately,
(10:28):
in practice, some fire investigators have been known to use
these highly faulty techniques that are not established to have
any scientific validity. It's just like we're saying earlier, kind
of like folk knowledge. The investigators pass it down, but
there's never been a study dedicated to figuring out whether
these rules of analysis are true or not. So in
(10:50):
a short article in September, the skeptic writer Michael Schermer
reported a conversation he had with a guy named John
Jay Lntini, who is a fire analysis expert and the
author of this this criminal this forensics textbook called Scientific
Protocols for Fire Investigation. So this is a guy who's
taking the scientific approach, and Lean Teeny tells him that
(11:11):
lots of fire investigators follow scientifically baseless folk wisdom, such
as that so called alligator ing can indicate whether a
fire burned fast or slow. And this alligator ing, this
this idea is that it's it's like looking at an
alligator's hide. So you look at wood that burned during
a fire, and if you look at it and you
(11:31):
see small flat blisters, that means that the fire burned
quote slow. And if you see large shiny blisters, that
means that it burned quote fast. Lntini his analysis of
this rule is quote nonsense. Yeah. Ok, that nobody's actually
measured that essentially, and then we quantified it and published
(11:53):
papers that we can refer back to. Right. Also, according
to len Tina, it used to be fire investing gator
wisdom that when you see crazing and window glass, you
know what crazing is. It's like when you see these, uh,
these sort of crazy spiderweb patterns virgin not exactly, not
like the circular spiderwebs, but just cracks all through the glass,
crazy all over the place. Right, This glass is now
(12:19):
legally insane. Yeah, So they uh, they used to say
that this was an indicator that the glass had been
heated very rapidly, which would indicate the use of an accelerant.
It heated rapidly because somebody squirted gasoline or lighter fluid
all over the place. In fact, that's not true, and
it turns out after scientific investigation that window crazing is
(12:40):
caused by the opposite. It's caused by rapid cooling, like
what would happen when firefighters douse the area with water. Okay,
I can picture this, yeah yeah. Um. Also another thing
that's common puddle shaped burns on the floor that can
make it look certainly like there was an ignited pool
of liquid on the floor, like somebody emptied a gas
can all over the place set it on fire. According
(13:02):
to Lenten E wrong again, even though fire and heat
do tend rise when an entire room burns, the floor
burns along with it, and these marks can appear without
the presence of an accelerant. Um. So, fire analysis is
one of the forensic fields addressed in this two thousand
nine in our C report, and they conclude that while
there is a fairly solid basis for explosion analysis like
(13:25):
when a bomb goes off, fire analysis is much shakier
than explosion analysis is as practice today. Um. They say,
quote many of the rules of thumb that are typically
assumed to indicate that an accelerant was used e g.
Alligatoring of wood specific char patterns have been shown not
to be true. Experiments should be designed to put arson
(13:46):
investigations on a more solid scientific footing. Uh. And again
this is just crazy. I mean, this is yet another
one of these that has been used to convict people,
like in the case of William So we don't know,
I guess whether he was guilty or innocent, But if
this was the main evidence to show he was guilty,
that's that makes it look pretty bad. Yeah, that's kind
(14:07):
of stuff is depressing, especially like when you I don't
know if you watched Making a Murderer, right, when you
watch these like long form documentaries about the justice system
and just how flawed it is. Not saying that in
that particular case, I know whether anybody is guilty or
innocent or anything, but it's you think about how often
this stuff goes on and then you've got examples like this.
It's heartbreaking. Uh. This section though, highlights one thing I
(14:30):
do want to emphasize again where we don't want to
paint all forensic science professionals and experts in these fields
with a with a you know, tainted brush, I mean,
and also saying here like most of them are doing
the best they can with what they have available to
him exactly, and in this case, the bad forensic science
and fire investigation is being exposed by good scientists and
fire investigation so it's not like everyone in this field
(14:52):
is doing a bad job of it, but many people
who are practicing it are. Yeah, this is truly a
wicked problem. I wish we had more money to divide
to divert into research and resources for stuff like this. Yeah. Well,
hair analysis is one of the ones we brought up
at the top because that was actually one of the
hair and fiber was one of the things we're talking about,
(15:13):
Josh about. Yeah, we're and we were also like, maybe
we'll do something like that in our little like video series,
and then the more we start to look into it,
we're like, I don't know about this, this is a
little hokey. Well okay, so one thing I can say is,
if you find a perpetrator's hair at the crime, couldn't
you use that to do some DNA analysis? That would
be pretty conclusive. I don't know, I would think so.
(15:35):
But again, remember the statistics on how often they use
DNA and say versus something like this, probably because of
the timing and the money involved. Yeah, um, whereas like
to just look at two fibers and have somebody come
in as an expert and say, yep, it's a match.
Seems easier to a lot of people, I would guess
are cheaper. Okay, what happens when there is hair matching
(15:58):
or hair fiber analysis. Well, I want to back up,
because there was just last year the Justice Department and
the FBI formally acknowledged that nearly every examiner in the
FBI's laboratory on microscopic hair comparison gave flawed testimony in
the trials that they offered evidence. And this is over
(16:20):
the course of two decades. This is huge. It's like
the biggest UH scandal in forensic science to date as
far as I can tell. Um. They were giving statements
that were quote, beyond the bounds of proper science. What
they were basically doing was saying there's a hundred percent
match between two hair fibers or a hair fiber found
(16:42):
on the scene the hair from you know, a defendant, uh,
when there's just a similarity. So they're totally misleading. The
jury's twenty six out of the twenty eight overstated forensic
mass matches favor the prosecutors, of course, because they're working together.
This equated to nine of the two hundred sixty trials.
(17:02):
According to the National Association of Criminal Defense lawyers and
the Innocence Project. This is the nation's largest post conviction
review of question forensic evidence. It's crazy. Of these cases,
thirty two defendants were sentenced to death. Fourteen have already
been executed or died in prison. So that's I mean,
(17:26):
whether they were innocent or guilty, we don't know, but
they don't yeah, and we'll never have a chance. Now
the cases against them have been undermined. And keep in
mind that just because the FBI made these errors, there
was other evidence used to convict these defendants of guilt. Okay,
so it wasn't the only thing. But still each case
(17:47):
has to be reviewed to see if there are grounds
for an appeal. Four were previously exonerated, So this is
a landmark revelation that pattern based forensic techniques, things like
hair bite mark analysis, ballistics largely subjective, and these contributed
to more than twenty five of the three hundred and
(18:08):
twenty nine cases where a defendant has been exonerated with
DNA analysis post conviction. So that's how DNA analysis has
been used very much in the last like ten to
twenty years. Well, we talked about the exclusionary principle earlier,
you know, a lot of what DNA evidence is actually
brought to the field is uh, not just matching and
saying here, you know this criminal, definitely, that's that's his
(18:29):
or her blood there, but excluding people saying, look, this
is not their day genetic material. Yeah, this is not
a match. So a lot of politicians weighed in on this.
Of course, they asked the FBI to do a systematic
analysis to breakdown their system, and the FBI says that
their hair examiners lacked written standards that defined what were
the appropriate ways to explain their results in court. That
(18:52):
was until two thousand and twelve. This all came to
light after the Washington Post reported on flawed forensic hair
match is. Federal authorities investigated this and they found it
to be true. They found that the experts quote unquote
that we're testifying to near certainty of matches in a
crime scene, they were citing incomplete and misleading statistics drawn
(19:14):
from their own work. So I want to make this clear,
there is no accepted research on how hair from different
people can appear the same. So, I mean, like you
asked at the topic, well, how does that work? I
mean I think it's literally like you look under a
microscope at the two pieces of hair, and that looks
similar to me. Yeah, And I mean I'm sure that
there's some more to it than that, right, in the
(19:36):
same way as like there's some old timey wisdom passed
down about how fires look the same, right, but there's
no accepted research that one one thing leads to another.
Even before this, the FBI reported it's examiners were reporting
false hair matches more than eleven percent of the time,
and at the time of this report, five defendants were
(19:58):
exonerated through d in a testing, all of whom had
served between twenty and thirty years in prison for either
rape or murder. Jeez, I mean, this is just, again,
like I said, heartbreaking. Of twenty one thousand federal or
state requests for hair comparison evidence from nineteen seventy two
to nineteen ninety nine, the FBI found that two thousand,
(20:20):
five hundred of those cases had examiners declare hair matches.
They're reviewing every single one of those cases now, like
they have to go back and just review everything. The
same examiners who are now under review also taught five
hundred to a thousand state and local crime analysis labs
(20:41):
how to testify in the same way. So this is
like the old timers passing down the wisdom, and so
it's an endemic problem. It's not just the FBI, it's
everywhere now. So these same testimonies were likely flawed at
the same level. It's just it's it's insane. In one
shocking example, this is just one example of how it
affected a person, a lot person's life. Sante Tribble served
(21:04):
twenty eight years for murder based on FBI hair testimony.
It later turned out that one of the hairs that
was used to prosecute him came from a dog. Oh
my god, he was exonerated in Wow, So hair analysis
not so much bite mark analysis. Okay, here's another example.
(21:28):
Expert in forensic odeontology, we mentioned that earlier testified that
multiple bite marks found on a murder victim were entirely
consistent with the dental impressions taken from a guy named
Roy Brown. This was the only physical evidence in this case.
Brown was sentenced to twenty five years to life in prison.
DNA tests later confirmed that a second suspect was actually guilty,
(21:50):
but Brown. He spent fifteen years in prison. So bite
mark analysis is now also widely considered unreliable. Yeah, it
was explored in the two is a nine in RC
document and that they outlined several problems with current use
of bite mark evidence. Uh, they say, quote, uniqueness of
human dentition has not been scientifically established. I mean you'd
(22:12):
think that that's pretty basic. You'd have to start with
the study making it statistically clear that humans have unique
bite marks. But I mean, what if they don't usually
have unique bite marks. I mean, I don't know what
you would expect bite marks to look like. Maybe lots
of people have very similar looking bite marks. And I
would also think that again, like this is I haven't
(22:34):
done research on this, but I would imagine that the
force of the bite would contribute to what the bite
marks look like. Well, that's another thing. So they say,
even if dentition is unique, even if everybody has a
unique bite mark in the same way that it's commonly
assumed people have unique fingerprints. Uh, the ability of the
(22:55):
person to transfer a unique pattern to human skin, and
then the ability of the skin to maintain the uniqueness
of that pattern has not been scientifically established. So, so
imagine just what happens when people are trying to analyze
a bite mark on skin that maybe sagging or distorting
the bites. The marks might not leave a clear impression
(23:17):
that's still there when the forensic scientist gets a picture
of it or gets to examine it. I don't even
know how you test this, like, well, I meanlock style,
Like you get like some molds of teeth and then
you take like a corpse and and have the molds
bite the corpse several times and see how they line up. Yeah,
I think you'd have to. You'd have to examine it
(23:37):
statistically with a large sample size, just testing different Yeah,
different molds of teeth or something like that. I don't know,
if you test on dead people's bodies or something like that. Well,
I don't know. I mean, it's the thing. I admit
this is difficult to test, but this is this is
ground level research that should be done if you're going
to be using this to convict people. Oh yeah, yeah,
yeah absolutely, or to defend people either way. All right,
(24:01):
so you're ready for the big one. Yeah, So DNA
analysis we established at the beginning, and I think this
is true, is widely considered, and I think with good reason,
the most reliable forensic science field. When you when you're
matching DNA evidence, you can know if the if the
procedures have been carried out carefully and and all of
the protocols have been followed, that if you have a
(24:23):
DNA sample of the crime scene, and you have a
DNA sample from the defendant, you can know, you know,
beyond a one in billions chance of error, that this
is the same person or it's not the same person. Yeah.
Except The Atlantic published an article this year called a
reasonable doubt. Uh, and it said exactly that it starts
(24:46):
off and saying yes. D NA analysis has long been
held as the exception to the rule about forensics. Arose
from academics. It's been studied and validated by researchers all
around the world. Uh, just a little history here. It
was pioneered by British geneticist named Alec Jeffreys. He was
looking into genetic sequencing and he applied it in the
(25:06):
field with police to help solve a pair of murders
in the British Midlands. Following that, several private companies in
the US and the UK opened their own forensic DNA labs.
But defense teams have argued against it for years. Well why, Well,
the first thing they said was, at first DNA analysis
actually didn't pass the fry test. I can understand when
(25:28):
it's new, it's not generally accepted yet. Yeah, they set
a legal standard requiring scientific evidence to have widespread acceptance
in his field was needed. Okay, well we have that now.
Academics complained that these firms weren't actually being transparent about
their testing technique. Well, our methodology wasn't. Well, that's a
perfectly acceptable concern. I think anybody doing this kind of
(25:49):
analysis should be making public exactly what their methodologies are.
And then, as we've well, I maybe not all of us.
I I was paying attention to the news at the time,
but as popularized by the O. J. Simpson trial, the
argument that DNA samples can in fact be contaminated during
both collection or in the crime lab. But throughout the years,
(26:12):
the thoroughness of labs and analysis list they all got better, right,
so so much so that the Innocence Project, which we've
been mentioning over and over again this episode, was founded
in because they were absolutely convinced that DNA evidence could
exonerate questionable convictions. They have since won one hundred and
seventy eight exonerations due to DNA testing. In a book
(26:35):
by the founders of the Innocence Project, they said, quote,
DNA testing is to justice what the telescope is for
the stars. But yeah, I think in many ways that's true.
And now we're gonna say what the flaws are. But overall,
DNA testing, I will agree, is super solid. Yeah, yeah, no,
I think you're right too. But it's important to sort
(26:56):
of lay this out to know that there are problems
with it. Uh So this has been amplified, of course
by the c s I effect that we've also been
talking about, right the expectation of jurors to see DNA
evidence in corn cases. In fact, in two thousand and eight,
there was a study done by a felony judge in
Michigan where he randomly pulled a thousand, twenty seven summon jurors,
(27:20):
and seventy five percent of them expected that they would
see DNA in a rape case. Fifty of them expected
that in a murder or an attempted murder case. They
would see DNA in expected they would see it in
any criminal case that they were a part of. The
DNA was just so widely available to research and use
as evidence that of course it would pop up. Yeah,
(27:41):
and this has got to be frustrating to I mean,
well meaning prosecutors. Who are you know, they're they're not
trying to cut corners or something, But in many cases
DNA evidence just might not be available. Well, and the
other side of it, too, is that DNA has become
such a powerful tool in the courtroom that it almost
automatically secures convictions, right because people hear those words d
(28:03):
N A and they're just like, well, that's irrefutable. Well,
I mean, I can say if if there's a murder
case and uh, and there are blood stains found on
a victim, and you can take those blood stains and
and several different DNA analysis labs, I'll say that it's
a perfect match for the defendant. I think it's pretty
likely that looks like guilt to me. You should watch
(28:25):
Making a Murderer Man, because there's a whole thing in
that about how like they found this guy's blood at
a at a crime scene in a car. And then
the argument is made, well, the police actually had access
to his blood from previous case. They may have planted
the blood there. Well, in that case, that's I would
say that's again not a problem with the scientific methodology,
but a problem with the with the investigation and the procedures. Yeah,
(28:49):
like you're right, so okay. Research from Australia found that
sexual assault cases involving DNA evidence were twice as likely
to reach trial and thirty three times is likely to
result in a guilty verdict. There's DNA right. Homicide cases
fourteen times is likely to reach trial, twenty three times
is likely to end in a guilty verdict. Another major
(29:11):
study in the UK found that just the knowledge that
prosecution intended to introduce DNA evidence before it was even
actually like introduced in a case was enough to get
a defendant to capitulate because they're just like, okay, so
there's questions about collection and storage, but they just they
just stopped because what do you mean stopped? Well, because
(29:32):
people were starting to think, like, well, this is just
irrefutable evidence. Why should we bother looking at the collection
and storage of it. DNA is just winning so many cases,
let's not bother with it. So as the cases we
use DNA evidence and become more complicated, it actually becomes
less of an objective science. And I'll give you an
example here so just real quick primer on this, nine
(29:55):
percent of our genes are the same as every other
human on the planet. My genes are the same as
Joe's jeans. But the DNA analysts know this, of course,
So we need to look for things that are called alleles.
They're very specific locations on each DNA strand that vary
from individual to individual, the different versions of a gene
(30:15):
in the different places on your chromosomes. The standard is
to compare alleles at thirteen locations. Now, if you do
it that way, the odds of two unrelated people matching
are less than one in a billion, So that's pretty good, right,
But what happens if you're looking at a case and
it involves a third person, So you're not just comparing
(30:36):
two DNA, you're comparing three, or what if you're comparing four?
What if there are four people at the scene. Figuring
out which alleles belong to whom becomes more complicated the
more people that are involved, And if a sample is
small or degraded, which they often are from crime scenes,
the alleles might drop out in some locations as well.
So there's a study done by a guy's name a
(30:58):
teal drawer and Greg hamp A key in they took
DNA paperwork from a two thousand to Georgia rape trial
here in the state that we're recording this in. They
gave the evidence to seventeen technicians without context. All of
these people were experienced DNA technicians with an average of
nine years in the field. They were asked to determine
(31:20):
if the DNA was from the defendant in the case.
Only one of the seventeen people concurred that it was
the defendant, twelve said it was exclusionary, for said it
was totally inconclusive. Since this example, there have been accounts
of dozens of DNA typing cases that have gone wrong,
especially because it's so easily contaminated depending on who comes
(31:43):
in contact with it. Right. So there's something that's called
d N A transfer which contributes to this the most.
It's when cells migrate from people to people or from
people to objects, inevitably when we just touched things. Right.
Another study asked participants to shake hands for two minutes
and hold a knife when the DNA on the knife
was analyzed. Well, to be a participant there just sitting
(32:08):
there holding the knife, and now you hold the knife.
What do I do with it? Just hold it? Just
hold it. Man. When the DNA on the knife was analyzed,
the partner was identified as a contributor in the cases
found them to be the sole contributor. They didn't even
find the DNA from the second person. So going forward,
more context needs to be around for this DNA analysis. Right.
(32:32):
The science is is there, but it it's complicated and
there needs to be like, I guess again, procedures put
in place for like, well, how how do you what?
What's the percentage of accuracy? I think is what we
need to come down on. Uh. Houston, Oh boy, Houston.
(32:53):
Houston had a real bad incident of this. Their crime
lab handled DNA evidence from five cases a year. A
local television station obtained dozens of the DNA profiles that
were processed by this lab and they sent them to
independent experts for analysis. It turned out that the technicians
were routinely misinterpreting even the most basic of DNA samples.
(33:15):
So this was just like a technician error, and the
Innocence Project themselves said, well, we don't take cases where
there are positive DNA matches. And one example, a tech
created a profile for a victim from three different sets
of DNA, the result profiles all totally varied. Then she
mismatched the DNA from the crime scene and the accused defendants.
(33:38):
This led to a retrial where the defendant was released.
So this tech was fired. This is the same Houston
lab she was fired. Then she was reinstated because her
lawyer said, well, the problem was systemic. It was the
whole lab in Houston that was the problem here. They
had inadequate supervision. Well that may have been the case,
it could have been. Yeah, but think about all five
(34:00):
of those cases. I mean, we think that this DNA
analysis is rock solid, and then you hear this and
it's like, well, there's five cases we have to go
back and review again. You know. So, oh man, this
is so crazy because it's you think of scientific evidence
as being the most reliable kind of evidence that you
(34:21):
can have in a courtroom, at least I would, I think.
But and and that's so scary that you often will
have this problem of not knowing whether you can trust
the expert witness telling you something because you don't know
if there if there was something wrong with the data
collection and contamination. You don't know if their methodologies are
actually sound to begin with, whether they've been based on
(34:45):
well tested scientific principles, do they have a clean lab.
Is their supervisor pushing them to do more work than
they're capable of doing exactly? Or is there is there
prosecutorial bias or something. Is there pressure from the dep
artment for them to uh basically convict more cases? Yeah?
(35:06):
And man, that's so tough because we've already seen evidence
in courtrooms undermined by lots of other kinds of studies.
I mean, the more we learn about the reliability of
eyewitness testimony, nothing ever makes it look more reliable. Uh. People,
it just turns out, man, people's memories are not very good.
(35:26):
And in many cases, people have they profess confidence in
identifying a you know, a perpetrator or something in a courtroom,
but then it turns out there wrong. And uh so,
so eyewitness testimony. What I'm not saying it's always wrong,
but it's just you can't be very confident in trusting it. Um.
(35:47):
We we've seen all these problems with forensic science and
so man, it is just it's just tough out there
to know that you're getting the right answer in a courtroom.
I would hate to be I would hate to be
the defendant in any case, especially a case in which
I was innocent. Right now, that's yecially if you're innocent, Yeah,
I think, yeah, it's guilty. I'd still hate it, but
(36:09):
I'd be like, but I did beat Joe down with
the folding chair, Yeah exactly, But how are they going
to tell bruise marks? There's no science that unless you're
Sherlock and you're beating corpses with a horse whip, like
I said, Uh, okay, jokes aside, Seriously, this is a
big problem. What do we do, like, what's the future
of this look like? How do we reform it? Well?
(36:31):
One of the problems is that forensic science encompasses many
different fields, right, so that lots all these fields have
different standards, and some have different problems. Some are more
solid than others, Some have more regulations in place to
keep all of the practitioners in line than others do. So,
so it's a it's a big, hairy, complicated problem, and
(36:53):
there's not one simple prescription that will fix it all. Yeah,
but there are some sort of general rules that some
fields may already be following pretty well, and others could
do a much better job of Yeah, some possibilities that
people are throwing out as they're saying, defendants should have
their own forensics experts that are paid for. Uh. There
should be separate crime labs from the prosecutors and the
(37:14):
police that they're answering to. And there should be, of course,
as we've been alluding to all episode, an established system
of verification and standards. Yeah. I mean one one hypothetical
you could imagine would be something like that. There is.
It would be sort of like the uh, forensic science
version of public financing of elections, where you say, you know,
(37:38):
like if you have mismatched resources going in, you can
make this fairer if you just give both sides equal
money from the government and that's what they have to
spend on elections. Uh. In in this case, you could say, well,
what if we just have a a crime lab that is,
you know, big national crime lab that works for neither
the defense nor the prosecution, and that in order to
(38:01):
get forensic science that's admissible in court, you have to
go through this this big uh, this big well regulated system,
and that that they don't work for either side. Yeah.
In fact, an independent entity that would be called the
National Institute of forensic science has been recommended for something
like this, and they would be responsible for both establishing
(38:23):
the standards that we're talking about and for certifying people
as experts. Another another way of helping the problem is
just from the ground up in each field itself. Like
we were talking about earlier in in fire analysis, they're
very legitimate scientists working in this field who are just
trying to improve the field by doing scientific research and
holding others in their field accountable to pay attention to it. Yeah, yeah,
(38:47):
that's true. Uh. There's also this one like case that
came up about DNA, which is interesting. There's a company
that's using software and automation to take the human element
out of it completely, and I've alluded to that as
well with like finger analysis things like that. Um, they're saying, well,
we take the subjective decisions out of it, and the
analyst is the one who's making the flawed conclusions. The
(39:09):
company calls itself true Allele uh. And the software, however,
has been criticized for whether it actually met the FRIES standard,
even though we're no longer using the price standard is
being criticized because it hasn't been accepted by the larger
science community. But one thing I would say about that
is I think if you have a computer program that's
that's doing analysis on forensic science, I think that would
(39:30):
need to be open source. And company who develops that
probably isn't gonna want that because they're gonna want it
to be proprietary own it. But that should be something
that anybody can go look at the source code and
make sure that it's working accurately and not biasing towards
a certain conclusion. Okay, so that was a lot about
(39:53):
forensic science, and unfortunately we don't this is already a
long episode. We don't have time to two more here,
and there were a bunch of other topics we wanted
to get into and just didn't even have space for,
like handwriting analysis, lie detection, the polygraph recovered memories. So
maybe we can address those. If you're interested, you can
write us and let us know you want to learn
(40:13):
about them, and we'll talk about them in a future episode. Yeah,
we can sit down record a future episode, maybe bring
in old Robert Lamb and uh we could uh um. Also,
like I would love to hear in the meantime from
some of you who perhaps work in this field, you know,
we have something to add to the discussion. Yes, is
there anything we missed out on or that's a really
interesting case of of of how these standards are enforced
(40:37):
in the field that you're familiar with. So if you
want to write into us about forensic science, you want
to talk to us about maybe your experiences, maybe you've
been on a jury like I have, or you work
in forensic science lab and you have some argument perhaps
with how we've portrayed the science here. Please let us know.
We'd love to hear from you. We are all over
(40:58):
the internet. You can find us on social media on Facebook, Twitter, Tumbler,
and Instagram now right. You can also visit us at
our home base, stuff to Blow your Mind dot com.
That's where you can find all of the podcasts we've done.
We also right there and all of our videos are
there as well. And of course, if you want to
email us and let us know your feedback on this
(41:19):
episode or any other, or just let us know a
topic you might like us to do in the future,
you can email us at blow the Mind at how
stuff works dot com for more on this and thousands
of other topics. Is that how stuff works dot com
Stuff To Blow Your Mind News
Hosts And Creators
Robert Lamb
Joe McCormick
Popular Podcasts
United States of Kennedy
United States of Kennedy is a podcast about our cultural fascination with the Kennedy dynasty. Every week, hosts Lyra Smith and George Civeris go into one aspect of the Kennedy story.
Stuff You Should Know
If you've ever wanted to know about champagne, satanism, the Stonewall Uprising, chaos theory, LSD, El Nino, true crime and Rosa Parks, then look no further. Josh and Chuck have you covered.
Dateline NBC
Current and classic episodes, featuring compelling true-crime mysteries, powerful documentaries and in-depth investigations. Follow now to get the latest episodes of Dateline NBC completely free, or subscribe to Dateline Premium for ad-free listening and exclusive bonus content: DatelinePremium.com