Over the course of this podcast, we've laid out not just the flaws of forensic science, but the horrific price people who are wrongly convicted pay for them, like spending decades in prison while the outside world moves on without you. Which Keith Harwood, whose story you heard in the Bite Marks episode, experienced.

Over the course of this podcast, we've laid out not just the flaws of forensic science, but the horrific price people who are wrongly convicted pay for them, like spending decades in prison while the outside world moves on without you. Which Keith Harwood, whose story you heard in the Bite Marks episode, experienced.

You're like in a coma.

You're like in a coma.

Now I was awoken, you know, April seventh, twenty sixteen, and now I got to learn what did I miss during that cola?

Now I was awoken, you know, April seventh, twenty sixteen, and now I got to learn what did I miss during that cola?

Separations that destroy marriages, like we heard in the case of Audrey Edmonds from the Shaken Baby Syndrome episode.

Separations that destroy marriages, like we heard in the case of Audrey Edmonds from the Shaken Baby Syndrome episode.

I'm very thankful now to have a wonderful boyfriend, to have a great contact and interaction with my children. I mean, I'm not happy. The scar in my life will always be there, but it fades.

I'm very thankful now to have a wonderful boyfriend, to have a great contact and interaction with my children. I mean, I'm not happy. The scar in my life will always be there, but it fades.

Loved ones who have passed funerals missed. Take Anthony Kyle's, who you heard about in the Arson Investigation episode. He was released in the fall of twenty twenty two after spending nearly half his life behind bars.

Loved ones who have passed funerals missed. Take Anthony Kyle's, who you heard about in the Arson Investigation episode. He was released in the fall of twenty twenty two after spending nearly half his life behind bars.

My mother and father they passed away in two thousand and five and two thousand and six, like six months apart to destroy my family.

My mother and father they passed away in two thousand and five and two thousand and six, like six months apart to destroy my family.

Some find ways to cope, like Charles Fain, who you heard from in the Footwear Analysis episode.

Some find ways to cope, like Charles Fain, who you heard from in the Footwear Analysis episode.

Nike the mest of it. That's what I did, thanks to Jesus and my faith in God.

Nike the mest of it. That's what I did, thanks to Jesus and my faith in God.

But the trauma doesn't go away. David Gavitt was wrongly convicted of arson and served nearly twenty seven years in prison.

But the trauma doesn't go away. David Gavitt was wrongly convicted of arson and served nearly twenty seven years in prison.

Inside prison, it's mass confusion. It's like a zoo. It's like a jungle in there, and it's noisy. I'm so used to being in quarters and around other men and being told what to do. Happen to ask permission to go to the bathroom? Can I go take a shower? Can I go to chaw now? I can remember my wife telling me, you don't have to ask my permission. David, you're free now.

Inside prison, it's mass confusion. It's like a zoo. It's like a jungle in there, and it's noisy. I'm so used to being in quarters and around other men and being told what to do. Happen to ask permission to go to the bathroom? Can I go take a shower? Can I go to chaw now? I can remember my wife telling me, you don't have to ask my permission. David, you're free now.

We simply can't know how many people are incarcerated in the United States right now for a crime they didn't commit based on faulty forensic science. But we can have hope for the future that things can change. Odell Adams has that hope. You met him in the Firearms Analysis episode. He's appealing his conviction and he has a message for those in prison for a crime they didn't commit.

We simply can't know how many people are incarcerated in the United States right now for a crime they didn't commit based on faulty forensic science. But we can have hope for the future that things can change. Odell Adams has that hope. You met him in the Firearms Analysis episode. He's appealing his conviction and he has a message for those in prison for a crime they didn't commit.

Tough times never last long.

Tough times never last long.

Tough people do.

Tough people do.

Just don't give up hope man at all. You know what I'm saying, There's always light at the end of the tunnel.

Just don't give up hope man at all. You know what I'm saying, There's always light at the end of the tunnel.

In our final episode, We're going to sit down with two experts you heard from throughout this podcast, Manca Sinha and Nicholas's Scourage, and talk about lessons learned and what comes next. I'm Molly Herman and this is CSI on trial.

In our final episode, We're going to sit down with two experts you heard from throughout this podcast, Manca Sinha and Nicholas's Scourage, and talk about lessons learned and what comes next. I'm Molly Herman and this is CSI on trial.

Two or three stays are really not enough to call something an impact spatter from gunshot that's going to put someone in prison the rest of your life.

Two or three stays are really not enough to call something an impact spatter from gunshot that's going to put someone in prison the rest of your life.

Show that making up a lie was gonna get you home sooner? What is it about a bite mark that we make a dentist, an expert in this area, see who shot at you?

Show that making up a lie was gonna get you home sooner? What is it about a bite mark that we make a dentist, an expert in this area, see who shot at you?

I did not.

I did not.

He said, I will sit in this jail and I will rot before I take a plea book.

He said, I will sit in this jail and I will rot before I take a plea book.

The problem with forensic science in the criminal legal system today is that it's an awful lot of forensic and not an awful lot of science.

The problem with forensic science in the criminal legal system today is that it's an awful lot of forensic and not an awful lot of science.

Episode eight. Light at the end of the Tunnel. Recently I sat down with Manka Sinha and nicholas Scourage. Manka spent a decade as a public defender in Washington, d c. She's now a law professor and teaches the Criminal Defense Clinic at the University of Maryland School of Law. She's published on how forensic science can contribute to wrongful convictions. Nicholas is a professor of psychology and criminology at the University of California, Irvine. He's analyzed dozens of studies related to forensic methods like firearms analysis, and he's been an expert witness at criminal trials. I started by asking them about the CSI effect, this idea that crime procedurals on television have created a false sense of trust around forensic science. Manka has seen it firsthand in her classrooms.

Episode eight. Light at the end of the Tunnel. Recently I sat down with Manka Sinha and nicholas Scourage. Manka spent a decade as a public defender in Washington, d c. She's now a law professor and teaches the Criminal Defense Clinic at the University of Maryland School of Law. She's published on how forensic science can contribute to wrongful convictions. Nicholas is a professor of psychology and criminology at the University of California, Irvine. He's analyzed dozens of studies related to forensic methods like firearms analysis, and he's been an expert witness at criminal trials. I started by asking them about the CSI effect, this idea that crime procedurals on television have created a false sense of trust around forensic science. Manka has seen it firsthand in her classrooms.

I teach a class on forensic evidence, and in the beginning of the semester, my students often come up to me and say, I'm taking this class because of CSI, and I'm just like, I love CSI so much, I'm so excited to learn about forensics in it. By the end of the semester, it's just it's a totally different viewpoint. I think that we've had this belief about CSI for so long. I mean, some of it is the stories that have been told to us by law enforcement and by prosecutors that have been sort of filtered out of courtrooms into the media, and then of course they are you know, sort of bolstered and maybe even bigger in our consciousness through shows like CSI and sort of media attention around around these things. We need to start poking holes in that kind of narrative and sort of recognizing that that narrative has been told to us for a reason.

I teach a class on forensic evidence, and in the beginning of the semester, my students often come up to me and say, I'm taking this class because of CSI, and I'm just like, I love CSI so much, I'm so excited to learn about forensics in it. By the end of the semester, it's just it's a totally different viewpoint. I think that we've had this belief about CSI for so long. I mean, some of it is the stories that have been told to us by law enforcement and by prosecutors that have been sort of filtered out of courtrooms into the media, and then of course they are you know, sort of bolstered and maybe even bigger in our consciousness through shows like CSI and sort of media attention around around these things. We need to start poking holes in that kind of narrative and sort of recognizing that that narrative has been told to us for a reason.

So, Nick, how do you explain to someone that we should be questioning the science that many Americans think is infallible. You look at these studies which are widely accepted as validation for these methods, and point out their flaws. But is it just happening in an echo chamber?

So, Nick, how do you explain to someone that we should be questioning the science that many Americans think is infallible. You look at these studies which are widely accepted as validation for these methods, and point out their flaws. But is it just happening in an echo chamber?

Nothing changes the academics writing about the flaws and the research, are you know, in one sense, kind of just talking to themselves. It really hasn't had an effect on judges in their gatekeeping function. And what's kind of striking to me, there were these reports by the National Academy of Sciences, a report by p CAST. I mean, these are very impressive, authoritative bodies of scientists, way above my pay grade, I would happily admit, and yet those reports, I think, for the most part, had very little effect on the way judges limit or exclude this type of testimony. And yet it's scientists going into court and you know, having to deal with the headache of that. It's not a pleasant experience. But actually going into court and talking about the studies has had some effect, and I think that's kind of one important lesson. I think the TIBs case and Firearms has kind of looked at as a case that kind of broke ground and change the trajectory even just a little bit. I was involved with that case, and I think really the success of that case can all be attributed to Maka. I mean, she was the attorney who litigated that case. Part of I think the issue is it's not just about me writing a critique or explaining the limitations of a study. It's having an attorney who can translate that into kind of language that's more palatable to judges to get them to kind of understand and see these issues.

Nothing changes the academics writing about the flaws and the research, are you know, in one sense, kind of just talking to themselves. It really hasn't had an effect on judges in their gatekeeping function. And what's kind of striking to me, there were these reports by the National Academy of Sciences, a report by p CAST. I mean, these are very impressive, authoritative bodies of scientists, way above my pay grade, I would happily admit, and yet those reports, I think, for the most part, had very little effect on the way judges limit or exclude this type of testimony. And yet it's scientists going into court and you know, having to deal with the headache of that. It's not a pleasant experience. But actually going into court and talking about the studies has had some effect, and I think that's kind of one important lesson. I think the TIBs case and Firearms has kind of looked at as a case that kind of broke ground and change the trajectory even just a little bit. I was involved with that case, and I think really the success of that case can all be attributed to Maka. I mean, she was the attorney who litigated that case. Part of I think the issue is it's not just about me writing a critique or explaining the limitations of a study. It's having an attorney who can translate that into kind of language that's more palatable to judges to get them to kind of understand and see these issues.

And just a reminder to our listeners, we talked about the Marquette Tibbs case in our Firearms Analysis episode. It was a case that set new precedent in challenging firearms analysis in court.

And just a reminder to our listeners, we talked about the Marquette Tibbs case in our Firearms Analysis episode. It was a case that set new precedent in challenging firearms analysis in court.

I'd love to add something to what Nick was saying. If you don't mind, Nick is illustrating or I think a really really important point. And I don't want to take credit for Tibbs. I was on a team of brilliant were continuing to do that work and are just absolutely brilliant when we talk to each other now, me and Nick and other people who are involved in that litigation. Nicol correct me if I'm wrong, but I think the first time that he testified in one of these cases was me doing his direct examination in tips and me asking questions and him answering them. And I think in the moment, I don't know that I felt it. I was concentrating on what I was going to ask next. But the takeaway, after thinking about it for so many years, is it was that dialogue that sort of having the judge tuned into exactly what he was saying in that moment, as opposed to the writing on the page that he's talking about that's been out there for so many years. That really opened eyes and really changed the way that courts are starting to think about these problems. So I think this idea of taking the academic literature and taking the study of these disciplines into the courtroom is a really important point and a really important sort of takeaway for those who think about these issues.

I'd love to add something to what Nick was saying. If you don't mind, Nick is illustrating or I think a really really important point. And I don't want to take credit for Tibbs. I was on a team of brilliant were continuing to do that work and are just absolutely brilliant when we talk to each other now, me and Nick and other people who are involved in that litigation. Nicol correct me if I'm wrong, but I think the first time that he testified in one of these cases was me doing his direct examination in tips and me asking questions and him answering them. And I think in the moment, I don't know that I felt it. I was concentrating on what I was going to ask next. But the takeaway, after thinking about it for so many years, is it was that dialogue that sort of having the judge tuned into exactly what he was saying in that moment, as opposed to the writing on the page that he's talking about that's been out there for so many years. That really opened eyes and really changed the way that courts are starting to think about these problems. So I think this idea of taking the academic literature and taking the study of these disciplines into the courtroom is a really important point and a really important sort of takeaway for those who think about these issues.

Nick, how do you explain to someone that we should be questioning the science that many Americans think is infallible.

Nick, how do you explain to someone that we should be questioning the science that many Americans think is infallible.

I had dinner with a close family friend who's a very experienced attorney, and I said, let's do a hypothetical, right, because all the law professors love hypotheticals. And I said, I've been reading studies on predictions made by astrologers. This one study, it's kind of a multiple choice exam where they look at items and then they pick which prediction is the most likely to be accurate. And I said, you know, it's amazing they complete these studies professional astrologers and they never commit any errors. And I said, doesn't that surprise you? And he said, oh yeah, there's got to be some sort of catch to this, right. I said, well, you know they in about half the questions they say I'm not going to respond, right, I don't know, And then of the half that they choose to respond to, lo and behold, they get all them right. And he says, well, yeah, I mean that's a stupid test. And by the way, did they bother giving this test to any sort of you know, non astrologer layman, And I said, no, why would they do that? They're experts. Well, there's some striking similarities between the hypothetical I describe for you and these firearms studies. And to be fair, the analogy is not perfect. But the bigger takeaway from it is if I frame this as astrology, which we all know is not science, it's not reliable. It's very easy to see the flaws of these studies and how the studies don't actually support the proposition that they're used to support. But as soon as I change that to firearms, right, something changes and you think, and I think there's a psychological explanation for this, But for some reason, it's like when you have a strong kind of prior belief about something and you're given evidence that conflicts with that prior belief. There's really kind of two things you can do. One you can update your prior belief, or two you can look for ways to kind of discredit that study.

I had dinner with a close family friend who's a very experienced attorney, and I said, let's do a hypothetical, right, because all the law professors love hypotheticals. And I said, I've been reading studies on predictions made by astrologers. This one study, it's kind of a multiple choice exam where they look at items and then they pick which prediction is the most likely to be accurate. And I said, you know, it's amazing they complete these studies professional astrologers and they never commit any errors. And I said, doesn't that surprise you? And he said, oh yeah, there's got to be some sort of catch to this, right. I said, well, you know they in about half the questions they say I'm not going to respond, right, I don't know, And then of the half that they choose to respond to, lo and behold, they get all them right. And he says, well, yeah, I mean that's a stupid test. And by the way, did they bother giving this test to any sort of you know, non astrologer layman, And I said, no, why would they do that? They're experts. Well, there's some striking similarities between the hypothetical I describe for you and these firearms studies. And to be fair, the analogy is not perfect. But the bigger takeaway from it is if I frame this as astrology, which we all know is not science, it's not reliable. It's very easy to see the flaws of these studies and how the studies don't actually support the proposition that they're used to support. But as soon as I change that to firearms, right, something changes and you think, and I think there's a psychological explanation for this, But for some reason, it's like when you have a strong kind of prior belief about something and you're given evidence that conflicts with that prior belief. There's really kind of two things you can do. One you can update your prior belief, or two you can look for ways to kind of discredit that study.

A drum beat in this series has always been well, once precedent is established, then you're enslaved by it. In a sense, and maybe precedent is also a fix because you get a TIBs, then maybe you get the next one in the next one, and that's established new precedent. So is that sort of the domino effect that could be part of what turns the ship around? You think, yeah, I.

A drum beat in this series has always been well, once precedent is established, then you're enslaved by it. In a sense, and maybe precedent is also a fix because you get a TIBs, then maybe you get the next one in the next one, and that's established new precedent. So is that sort of the domino effect that could be part of what turns the ship around? You think, yeah, I.

Think it could be. I mean, I think that precedent is, as you're describing, a sort of quite intractable and sort of difficult problem because there's a lot of precedent that's not really based on any critical analysis of a discipline. And then just as you're saying there's domino effect, there's one case that doesn't look closely at the science behind a method and lets it in, and then another case said, well, this first case let it in, and then that goes on and on and on. It's a tough problem. I do think that these cases in which courts are starting to pay attention are one way in which the momentum can change, or at least we can sort of slow the domino effect. I also think for courts and judges who care about these issues, and there are many, they also provide a little bit of a roadmap of how can I sort of look at what the precedent was and say that's problematic, and Nick was talking about tips. I think the judge in that case did a very nice job of that, of sort of going back and looking at every case in which there was some sort of admission and saying, well, this case didn't really actually do an analysis. This case didn't do a comprehensive examination of what the discipline is. And so I think that you're also giving courts a little bit of an idea of how to approach these questions. Precedent is tough, and I think it's going to be it's going to continue to be sort of a difficult problem because of just the way the sort of legal system works. But I think there's a little bit of a pathway there.

Think it could be. I mean, I think that precedent is, as you're describing, a sort of quite intractable and sort of difficult problem because there's a lot of precedent that's not really based on any critical analysis of a discipline. And then just as you're saying there's domino effect, there's one case that doesn't look closely at the science behind a method and lets it in, and then another case said, well, this first case let it in, and then that goes on and on and on. It's a tough problem. I do think that these cases in which courts are starting to pay attention are one way in which the momentum can change, or at least we can sort of slow the domino effect. I also think for courts and judges who care about these issues, and there are many, they also provide a little bit of a roadmap of how can I sort of look at what the precedent was and say that's problematic, and Nick was talking about tips. I think the judge in that case did a very nice job of that, of sort of going back and looking at every case in which there was some sort of admission and saying, well, this case didn't really actually do an analysis. This case didn't do a comprehensive examination of what the discipline is. And so I think that you're also giving courts a little bit of an idea of how to approach these questions. Precedent is tough, and I think it's going to be it's going to continue to be sort of a difficult problem because of just the way the sort of legal system works. But I think there's a little bit of a pathway there.

What do you guys think about the huge army of practitioners out there. We talked to practitioners for this series, and I think they all definitely believe that they're on the side of justice.

What do you guys think about the huge army of practitioners out there. We talked to practitioners for this series, and I think they all definitely believe that they're on the side of justice.

I mean, it's obviously tough. One case in which I testified the opposing expert, who's a very friendly man who's been doing this for over fifty years, and I think he truly believes that he can reliably do what he has been doing for fifty years, and so someone like me coming in and saying, oh, well, you know, you haven't actually proven that you can do what you say you can do. I mean, it's not surprising they're not going to go, oh, yeah, you're right. I've spent half a century on this and it's all a sham. So of course it's not surprising that the examiners are not very receptive to all of these criticisms.

I mean, it's obviously tough. One case in which I testified the opposing expert, who's a very friendly man who's been doing this for over fifty years, and I think he truly believes that he can reliably do what he has been doing for fifty years, and so someone like me coming in and saying, oh, well, you know, you haven't actually proven that you can do what you say you can do. I mean, it's not surprising they're not going to go, oh, yeah, you're right. I've spent half a century on this and it's all a sham. So of course it's not surprising that the examiners are not very receptive to all of these criticisms.

I couldn't agree more with Nick that it's not surprising that there's some reluctance. There's some resistance. You can understand why. I mean, if somebody is dedicated their entire career, decades and decades to thinking a particular way and doing a particular thing, it's difficult to hear otherwise. And to your earlier point about how people who really know what they're doing can still make mistakes. People who know how to look under a microscope and look at find details don't necessarily know how to conduct a study that's going to ensure the reliability or the validity of a discipline. But those are not the same skill set. And in terms of the question about justice and believing that they do justice, I think they do believe that they do justice. I just think that one of the things we have to recognize individually but also as a culture, in a society as a whole, is that there are different conceptions of justice. One of the things that we talked about in the series is that most of these disciplines came out of police labs and came, like we're built off of an express desire to increase law enforcement ability and to pursue suspects and to earn convictions. And there's a well known, like a well studied type of bias that can infect your thinking when you're hired by a particular side, or when your whole discipline, or when you work exclusively almost with one particular side of the legal system. And to me, we need to have sort of a bigger way of thinking about justice. Justice is about fairness and process, and justice is about making sure that everybody gets a fair shake and making sure that the evidence that's presented against somebody is reliable and valid.

I couldn't agree more with Nick that it's not surprising that there's some reluctance. There's some resistance. You can understand why. I mean, if somebody is dedicated their entire career, decades and decades to thinking a particular way and doing a particular thing, it's difficult to hear otherwise. And to your earlier point about how people who really know what they're doing can still make mistakes. People who know how to look under a microscope and look at find details don't necessarily know how to conduct a study that's going to ensure the reliability or the validity of a discipline. But those are not the same skill set. And in terms of the question about justice and believing that they do justice, I think they do believe that they do justice. I just think that one of the things we have to recognize individually but also as a culture, in a society as a whole, is that there are different conceptions of justice. One of the things that we talked about in the series is that most of these disciplines came out of police labs and came, like we're built off of an express desire to increase law enforcement ability and to pursue suspects and to earn convictions. And there's a well known, like a well studied type of bias that can infect your thinking when you're hired by a particular side, or when your whole discipline, or when you work exclusively almost with one particular side of the legal system. And to me, we need to have sort of a bigger way of thinking about justice. Justice is about fairness and process, and justice is about making sure that everybody gets a fair shake and making sure that the evidence that's presented against somebody is reliable and valid.

I think if you were to just listen to this podcast, you might think that we cherry picked a few wrongful conviction cases that are just unicorns. Of course, any exoneration is kind of a unicorn, because they're very hard to achieve. But can you give us any sense of scale, you know, in terms of how many innocent people are convicted in America? Or is that like a complete mystery.

I think if you were to just listen to this podcast, you might think that we cherry picked a few wrongful conviction cases that are just unicorns. Of course, any exoneration is kind of a unicorn, because they're very hard to achieve. But can you give us any sense of scale, you know, in terms of how many innocent people are convicted in America? Or is that like a complete mystery.

No, it's not a complete mystery. I think setting the table a little bit is important. So, wrongful convictions occur primarily, if not near exclusively, in cases in which there's a trial record, so in cases that went to trial. So we have to start by thinking about how many cases didn't go to trial. And in the criminal legal system, ninety five percent plus cases never go to trial because most cases get resolved by a plea agreement. And so when you're thinking about wrongful convictions as unicorns, you're thinking of the tiniest, tiniest slice of those trial cases where an innocence claim not only is raised but heard and then is successful, which, as you just described, is an immense challenge in and of itself. But that's the tiniest slice of the tiniest slice of the pie. And there are a lot of reasons that people who are one hundred percent innocent plead guilty anyway, And the most obvious reason is I'm charged with the crime in which I'm facing a life sentence or in some cases of the death penalty, and I get a plea offer that says, well, you can do ten years or fifteen years, and that will be absolutely life altering too, but it's worth the risk of avoiding that much much, much worse sentence. And so wrongful convictions are extremely small, small number of cases. You can feel like, because we have a lot of attention around innocence cases that they happen all the time, but they really don't. And so for every person who successfully is exonerated, there are countless more languishing in prisons around the country who will never find somebody to even take their innocence claim, let alone raise it, let alone have a court hear it, let alone go through the multiple layers of steps that one has to go through to actually be successful.

No, it's not a complete mystery. I think setting the table a little bit is important. So, wrongful convictions occur primarily, if not near exclusively, in cases in which there's a trial record, so in cases that went to trial. So we have to start by thinking about how many cases didn't go to trial. And in the criminal legal system, ninety five percent plus cases never go to trial because most cases get resolved by a plea agreement. And so when you're thinking about wrongful convictions as unicorns, you're thinking of the tiniest, tiniest slice of those trial cases where an innocence claim not only is raised but heard and then is successful, which, as you just described, is an immense challenge in and of itself. But that's the tiniest slice of the tiniest slice of the pie. And there are a lot of reasons that people who are one hundred percent innocent plead guilty anyway, And the most obvious reason is I'm charged with the crime in which I'm facing a life sentence or in some cases of the death penalty, and I get a plea offer that says, well, you can do ten years or fifteen years, and that will be absolutely life altering too, but it's worth the risk of avoiding that much much, much worse sentence. And so wrongful convictions are extremely small, small number of cases. You can feel like, because we have a lot of attention around innocence cases that they happen all the time, but they really don't. And so for every person who successfully is exonerated, there are countless more languishing in prisons around the country who will never find somebody to even take their innocence claim, let alone raise it, let alone have a court hear it, let alone go through the multiple layers of steps that one has to go through to actually be successful.

One of the things that I think people will say, well, without these tools, right, what's left? How do we catch bad guys? Now? I do understand that a tool that sends you in the wrong direction means the bad guy doesn't get caught. You know, what do you say to sort of that investigative question?

One of the things that I think people will say, well, without these tools, right, what's left? How do we catch bad guys? Now? I do understand that a tool that sends you in the wrong direction means the bad guy doesn't get caught. You know, what do you say to sort of that investigative question?

For me, it's really just a due process issue. It's not fair to use evidence that is at bottom false. And this is not to say that fire arm examiners have nothing to offer. I think they should be allowed to come into court and testify about class characteristics. That's very probative evidence. It's very helpful to the prosecution. Where I object at this point, based on the evidence base, is going further than that and saying these two bullets definitively came from the same gun. In a not to dis impass, the examiners were adding an additional statement onto that, which is to the exclusion of all other guns on earth, which is not scientific. It's never been demonstrated. I don't even know how you could demonstrate that. I think one thing. In America, we try to pride ourselves on following a proper and legitimate procedure, and in theory, following a proper and legitimate procedure will make the whole system be perceived as more legitimate. I want to be clear that I'm not implying that the average firearmy examiner is consciously doing this sort of thing. I think it's more of a kind of cog and the system type of issue. But for me, it's just a simple due process issue. Let's make sure the process is fair and legitimate, that these defendants are given every sort of constitutional right and safeguard that they're afforded, and then if they're convicted, I don't really have any qualms about it, just as long as a procedure is proper.

For me, it's really just a due process issue. It's not fair to use evidence that is at bottom false. And this is not to say that fire arm examiners have nothing to offer. I think they should be allowed to come into court and testify about class characteristics. That's very probative evidence. It's very helpful to the prosecution. Where I object at this point, based on the evidence base, is going further than that and saying these two bullets definitively came from the same gun. In a not to dis impass, the examiners were adding an additional statement onto that, which is to the exclusion of all other guns on earth, which is not scientific. It's never been demonstrated. I don't even know how you could demonstrate that. I think one thing. In America, we try to pride ourselves on following a proper and legitimate procedure, and in theory, following a proper and legitimate procedure will make the whole system be perceived as more legitimate. I want to be clear that I'm not implying that the average firearmy examiner is consciously doing this sort of thing. I think it's more of a kind of cog and the system type of issue. But for me, it's just a simple due process issue. Let's make sure the process is fair and legitimate, that these defendants are given every sort of constitutional right and safeguard that they're afforded, and then if they're convicted, I don't really have any qualms about it, just as long as a procedure is proper.

We really need to start reminding ourselves that the question is not how do we get the bad guy? The question is the process fair? Is a process fair for everybody? You can't have faith in a conviction. You can't have faith in the process. You can't have faith that fairness occurred if you're using evidence as not reliable. So that's just like the baseline, even before you get to the question of like, okay, what will we do now? So that's the baseline. If it's not reliable, we shouldn't use it period for any purpose against any single person who's accused. But what can prosecutors use I mean, let's remind ourselves of who prosecutors are, right, Like, they are the most powerful actors in the system. They have the weight of law enforcement behind them, and the weight of that like comes with extraordinary investigative power. So think about all of the things that they've traditionally done like interviewing eyewitnesses who saw things that happen right. They can still do that. They can still observe suspects, You can develop a suspect. You can still watch their actions for hours, days, week, weeks. You can see who they're calling, you know, you can go to the phone company and get their records. You can look at what they're writing online. You can look at their criminal history for patterns and the way they're doing things. You can do tons and tons of different things. I think the question that we're raising is what's reliable and what's not reliable?

We really need to start reminding ourselves that the question is not how do we get the bad guy? The question is the process fair? Is a process fair for everybody? You can't have faith in a conviction. You can't have faith in the process. You can't have faith that fairness occurred if you're using evidence as not reliable. So that's just like the baseline, even before you get to the question of like, okay, what will we do now? So that's the baseline. If it's not reliable, we shouldn't use it period for any purpose against any single person who's accused. But what can prosecutors use I mean, let's remind ourselves of who prosecutors are, right, Like, they are the most powerful actors in the system. They have the weight of law enforcement behind them, and the weight of that like comes with extraordinary investigative power. So think about all of the things that they've traditionally done like interviewing eyewitnesses who saw things that happen right. They can still do that. They can still observe suspects, You can develop a suspect. You can still watch their actions for hours, days, week, weeks. You can see who they're calling, you know, you can go to the phone company and get their records. You can look at what they're writing online. You can look at their criminal history for patterns and the way they're doing things. You can do tons and tons of different things. I think the question that we're raising is what's reliable and what's not reliable?

After the break some advice if you're ever asked to serve on a jury and evaluate forensic science. We've outlined a lot of the structural and societal issues that have led to unscientific forensic science disciplines. But what should we do with them? Manca wants us to think big.

After the break some advice if you're ever asked to serve on a jury and evaluate forensic science. We've outlined a lot of the structural and societal issues that have led to unscientific forensic science disciplines. But what should we do with them? Manca wants us to think big.

I think the question is not what do we do with them, but rethinking our entire approach to the way we do business in the criminal legal system, particularly with these forensic methods, and sort of engaging in a real confrontation with whether or not we should be using them at all, and instead of thinking about salvaging, thinking about what are the definitively reliable and valid uses, if any, and limiting to those uses specifically. So I think it's a much bigger question. Then what do we do with them moving forward? Should we be using them at all? And if so, what are the very specific ways we can be confident about reliability.

I think the question is not what do we do with them, but rethinking our entire approach to the way we do business in the criminal legal system, particularly with these forensic methods, and sort of engaging in a real confrontation with whether or not we should be using them at all, and instead of thinking about salvaging, thinking about what are the definitively reliable and valid uses, if any, and limiting to those uses specifically. So I think it's a much bigger question. Then what do we do with them moving forward? Should we be using them at all? And if so, what are the very specific ways we can be confident about reliability.

Can you guys talk about the need for a federal agency, which was the recommendation of p CAST, and is that something that could help with all of this or that you guys feel is one potential step forward.

Can you guys talk about the need for a federal agency, which was the recommendation of p CAST, and is that something that could help with all of this or that you guys feel is one potential step forward.

It's very complicated, there's no doubt about that. I mean, on the one hand, it's striking to me that the FDA and the CDC regulate what sort of pharmaceuticals can go on the market, and they require very specific benchmarks to be met. They require a very specific type of study to be conducted, and we don't have anything like that in the forensics domain. And in fact, I sometimes tell judges when these emergency COVID tests were being created very rapidly to go on the market, the CDC still set benchmarks that certain test metrics had to be met, otherwise the test was not sufficiently valid to be used to test for COVID. Even if we buy the numbers from the firearm studies, they're not meeting those benchmarks. And so it's kind of striking to me that what are the consequences of using an invalid COVID test. They're not great, to be sure, but they're certainly I think less consequential for the most part. I'm sure there are some extreme examples then using an invalid technique that could potentially send someone to prison for life. On the other hand, what sort of authority would this regulatory agency actually have over small, independent state operated crime labs. I mean, could they really compel these crime labs not to do certain things or to follow a certain protocol? I don't think that that's very likely. And the flip side to this are these kind of half measures. So we do have some sort of accrediting bodies and regulatory agencies that exist and have existed. These are things that examiners go into court and proudly tout, you know, my crime lab is accredited. We follow their rules, their procedures. Other labs across America follow these Judges are able to check that box that they're following accepted standards, and we saw how that turned out. I don't see an easy answer. I think anyone who says there is a very simple way to do this is fooling themselves.

It's very complicated, there's no doubt about that. I mean, on the one hand, it's striking to me that the FDA and the CDC regulate what sort of pharmaceuticals can go on the market, and they require very specific benchmarks to be met. They require a very specific type of study to be conducted, and we don't have anything like that in the forensics domain. And in fact, I sometimes tell judges when these emergency COVID tests were being created very rapidly to go on the market, the CDC still set benchmarks that certain test metrics had to be met, otherwise the test was not sufficiently valid to be used to test for COVID. Even if we buy the numbers from the firearm studies, they're not meeting those benchmarks. And so it's kind of striking to me that what are the consequences of using an invalid COVID test. They're not great, to be sure, but they're certainly I think less consequential for the most part. I'm sure there are some extreme examples then using an invalid technique that could potentially send someone to prison for life. On the other hand, what sort of authority would this regulatory agency actually have over small, independent state operated crime labs. I mean, could they really compel these crime labs not to do certain things or to follow a certain protocol? I don't think that that's very likely. And the flip side to this are these kind of half measures. So we do have some sort of accrediting bodies and regulatory agencies that exist and have existed. These are things that examiners go into court and proudly tout, you know, my crime lab is accredited. We follow their rules, their procedures. Other labs across America follow these Judges are able to check that box that they're following accepted standards, and we saw how that turned out. I don't see an easy answer. I think anyone who says there is a very simple way to do this is fooling themselves.

Manca, what do you think?

Manca, what do you think?

I struggle with the question about a federal agency for many years. I one hundred percent agreed with the two thousand and nine NAS report in that recommendation for sort of central national forensic overt body. I thought it was a great idea of very necessary reform. But the longer I've been immersed in these issues, the less I am convinced about that, and the less optimistic I am about that. And not to say that I am sort of abandoning that idea kind of wholesale, but I have a lot of questions about it because one of the things that we've seen is the resistance in the forensic and law enforcement communities towards change. And there was an effort in the wake of that report, in the early years after that report to sort of create that type of a body, and there was a lot of pushback. There was a tremendous amount of pushback from DJ and from law enforcement institutions, and you know it as we know, No, it never happened, and I worry that there's a mentality, at least in some segments of the discipline that is sort of you know, sort of comfortable with things as a status quo and things the way they are now. When you zoom out and you just look at the criminal legal system, what does forensics do. You've got a system in which black people and brown people and other and indigenous people and other marginalized people. They are arrested more, they are detained before trial, more, they are given worse plea offers. They are convicted more, they're sentenced to lengthier sentences, they are wrongfully convicted more, all disproportionately more, all controlling for other factors now fold in forensics. Right, they enable police to do all of those things that I just talked about, Right, So they enable you to ensnare people into this really brutal, this really punitive system. They enable prosecutors to make their case. So when you start to see how they can influence and reinforce this cycle, you can see how they are all connected. Right, when we think about wrongful convictions and how they disproportionately affect black people, well, forensics is part of that. You can't just strip it out and treat it like its own little bubble. It's all connected, it's all interrelated, it's part and parcel.

I struggle with the question about a federal agency for many years. I one hundred percent agreed with the two thousand and nine NAS report in that recommendation for sort of central national forensic overt body. I thought it was a great idea of very necessary reform. But the longer I've been immersed in these issues, the less I am convinced about that, and the less optimistic I am about that. And not to say that I am sort of abandoning that idea kind of wholesale, but I have a lot of questions about it because one of the things that we've seen is the resistance in the forensic and law enforcement communities towards change. And there was an effort in the wake of that report, in the early years after that report to sort of create that type of a body, and there was a lot of pushback. There was a tremendous amount of pushback from DJ and from law enforcement institutions, and you know it as we know, No, it never happened, and I worry that there's a mentality, at least in some segments of the discipline that is sort of you know, sort of comfortable with things as a status quo and things the way they are now. When you zoom out and you just look at the criminal legal system, what does forensics do. You've got a system in which black people and brown people and other and indigenous people and other marginalized people. They are arrested more, they are detained before trial, more, they are given worse plea offers. They are convicted more, they're sentenced to lengthier sentences, they are wrongfully convicted more, all disproportionately more, all controlling for other factors now fold in forensics. Right, they enable police to do all of those things that I just talked about, Right, So they enable you to ensnare people into this really brutal, this really punitive system. They enable prosecutors to make their case. So when you start to see how they can influence and reinforce this cycle, you can see how they are all connected. Right, when we think about wrongful convictions and how they disproportionately affect black people, well, forensics is part of that. You can't just strip it out and treat it like its own little bubble. It's all connected, it's all interrelated, it's part and parcel.

Let's end on some practical advice for our listeners. Say you are called to jury duty in a case that involves forensic science, what do you recommend?

Let's end on some practical advice for our listeners. Say you are called to jury duty in a case that involves forensic science, what do you recommend?

Oh, that's a real easy question to answer, Just kidding. When you're a jury you're taking an oath to consider the evidence that's presented to you. With science is critically important, at least forensic science, to try and understand what the experts are telling you in that particular case and not rely on your background beliefs about what you saw in a movie that looked really cool and seems sintuitively very plausible. That's part of the issue with firearms is that when they say each barrel leaves a fingerprint on a bullet, I think a lot of people find that easy to believe. And then you're sitting there as a juror, and there's other evidence that is pointing towards a defendant's guilt, and you have this scientist who says, oh, you know, the barrels leave unique marks on bullets, and I've determined it to match. It's very easy to kind of uncritically accept that, And so my advice would be try to look at the evidence individually and really interrogate, you know, the basis of your belief for accepting it.

Oh, that's a real easy question to answer, Just kidding. When you're a jury you're taking an oath to consider the evidence that's presented to you. With science is critically important, at least forensic science, to try and understand what the experts are telling you in that particular case and not rely on your background beliefs about what you saw in a movie that looked really cool and seems sintuitively very plausible. That's part of the issue with firearms is that when they say each barrel leaves a fingerprint on a bullet, I think a lot of people find that easy to believe. And then you're sitting there as a juror, and there's other evidence that is pointing towards a defendant's guilt, and you have this scientist who says, oh, you know, the barrels leave unique marks on bullets, and I've determined it to match. It's very easy to kind of uncritically accept that, And so my advice would be try to look at the evidence individually and really interrogate, you know, the basis of your belief for accepting it.

If you're a jur and you're sitting in a criminal case, I think there's just one very clear, easy thing you can do if you're in a case where you're presented with some sort of forensic evidence, and that's just be critical. Sort of put on your like science hat, think about all of the things you learned and whatever the last science class you took, and just be critical. Don't take what you're being told for granted, and think about whether or not it really makes sense. And if you hear a phrase like to a reasonable degree of scientific certainty or to a reasonable degree of insert discipline here certainty, remember that doesn't make any sense. That's a made up phrase for courtroom purposes only, and it has no true meaning. But the real thing you can do as a jur is sort of remember which you sort of learned here about how all of these disciplines have their limits, and many of them have very limited reliability as a sort of baseline, And you don't have to throw that out when you walk in the courtroom doors. You get to bring that experience and that knowledge and that understanding with you and you get to apply it to the case that's in front of you, and so that the main takeaway for a jur is just is be critical.

If you're a jur and you're sitting in a criminal case, I think there's just one very clear, easy thing you can do if you're in a case where you're presented with some sort of forensic evidence, and that's just be critical. Sort of put on your like science hat, think about all of the things you learned and whatever the last science class you took, and just be critical. Don't take what you're being told for granted, and think about whether or not it really makes sense. And if you hear a phrase like to a reasonable degree of scientific certainty or to a reasonable degree of insert discipline here certainty, remember that doesn't make any sense. That's a made up phrase for courtroom purposes only, and it has no true meaning. But the real thing you can do as a jur is sort of remember which you sort of learned here about how all of these disciplines have their limits, and many of them have very limited reliability as a sort of baseline, And you don't have to throw that out when you walk in the courtroom doors. You get to bring that experience and that knowledge and that understanding with you and you get to apply it to the case that's in front of you, and so that the main takeaway for a jur is just is be critical.

I think Nick and Manka, thank you both so much for coming in and talking through this. Your perspectives are so useful and even make me feel cast optimistic. I really appreciate it.

I think Nick and Manka, thank you both so much for coming in and talking through this. Your perspectives are so useful and even make me feel cast optimistic. I really appreciate it.

Thank you so much for having us and for taking on this issue that is near and dear to our hearts.

Thank you so much for having us and for taking on this issue that is near and dear to our hearts.

Thanks for having us.

Thanks for having us.

That's the end of the series for us. But before we go, I want to give us special thanks to every person who allowed us into their homes to relive some of the worst moments of their lives. They did not receive any compensation for being on this show. Teresa Banner is one of my very favorite people I've met on this project. When we interviewed her, she sent me home with a snuggie because I'd admired hers. She manages a wingstop in North Carolina and her two sons are in prison for life for a murder they didn't commit. Keith Harward, David cam Charles Fain, Audrey Edmonds, David Gavitt, Anthony Kyles. Each one, in their own words, gave a single reason for participating to shed light on this problem so our country can become a more just and safe place. I thank them for their selflessness and hope we have done right by them. CSI on Trial is a co production of iHeart Podcasts and School of Humans, based on the Curiosity stream series CSI on Trial, created by Eleanor Grant and produced by the Biscuit Factory. You can watch all six episodes of the video series right now at CuriosityStream dot com. This episode is hosted and written by me Molly Herman and researched by Katie Dunn and myself. Our producer is Miranda Hawkins. Jessica Metzker is the senior producer. Virginia Prescott, Jason English, Brandon Barr and L. C. Crowley are the executive producers. Sound design and mix by Miranda Hawkins. Special thanks to John Higgins, Rob Burke, Rob Lyle, and Brandon Craigie. If you're enjoying the show, leave a review in your favorite podcast app. Check out the Curiosity Audio Network for podcasts covering history, pop culture, true crime, and more.

That's the end of the series for us. But before we go, I want to give us special thanks to every person who allowed us into their homes to relive some of the worst moments of their lives. They did not receive any compensation for being on this show. Teresa Banner is one of my very favorite people I've met on this project. When we interviewed her, she sent me home with a snuggie because I'd admired hers. She manages a wingstop in North Carolina and her two sons are in prison for life for a murder they didn't commit. Keith Harward, David cam Charles Fain, Audrey Edmonds, David Gavitt, Anthony Kyles. Each one, in their own words, gave a single reason for participating to shed light on this problem so our country can become a more just and safe place. I thank them for their selflessness and hope we have done right by them. CSI on Trial is a co production of iHeart Podcasts and School of Humans, based on the Curiosity stream series CSI on Trial, created by Eleanor Grant and produced by the Biscuit Factory. You can watch all six episodes of the video series right now at CuriosityStream dot com. This episode is hosted and written by me Molly Herman and researched by Katie Dunn and myself. Our producer is Miranda Hawkins. Jessica Metzker is the senior producer. Virginia Prescott, Jason English, Brandon Barr and L. C. Crowley are the executive producers. Sound design and mix by Miranda Hawkins. Special thanks to John Higgins, Rob Burke, Rob Lyle, and Brandon Craigie. If you're enjoying the show, leave a review in your favorite podcast app. Check out the Curiosity Audio Network for podcasts covering history, pop culture, true crime, and more.

School of Humans,

School of Humans,