Oh can you name this TV show? So the width of the burn and the pinpoint detail in the fuel, I've narrowed down what our suspects using. Burned the tags off a blowtworks. Yeah, burns a small precise, just like what you get with compact torches used in dental labs. It's CSI Crime Scene Investigation, a CBS crime drama featuring a team of investigators solving crimes by analyzing the physical evidence. See once a dried, the pattern revealed itself. You see this discoloration right here. Looks at the tread from a boot well I initially thought was a V is actually a WU And that's the logo for Woodridge Hunting boots. They look at things like bloodstain patterns, footprints, and bullet casings. The original show launched in two thousand, then came the spin and offs, inspiring still more shows that dig into forensic science, Bones and CIS Dexter. The list is long. These crime scene procedurals. They're fun, they make investigations digestible, engaging millions watch them. But these shows also propel the idea that forensic science is precise in proving guilt or innocence. It's called the CSI effect. The CSI effect, from a defense attorney's perspective is is that and shows like that they depict forensic evidence as infallible, and that it's quick, it's fast, and they do it in a half an hour, and they wrap it all up right, and that jurors come into lajers come into a criminal case with the expectation that forensic evidence can conceivably do that. That's Chris fabricand I'm the director of Strategic Litigation at the Innocence Project in New York, and i am the author of Junk Science and the American Criminal Justice System. CSI and shows like it give jurors a false sent of security that forensic science methods are totally accurate, that expert testimony from CSI analysts is infallible. Studies show that juries put an outsized amount of trust in the testimony of experts. But as we'll discuss in this podcast, forensic science methods and the expert testimony that shares that evidence with juries can both be flawed. It's an issue that has serious consequences. CSI this all crap. That's Keith Harward, one of several people you'll meet in this series, who was wrongly convicted based on forensic evidence that turned out to be wrong. I'm Molly Herman. I'm a filmmaker and producer of a six part documentary series for Curiosity Stream that digs into forensic crime science, one discipline at a time. I've interviewed nearly seventy practitioners, attorneys, scientists, journalists, judges, and innocent people who spend decades in prison, some who are still there. When I started this project, I believed that old adage that the US justice system isn't perfect, but it's still the best system out there. But what I discovered is that many methods of forensic science used to build criminal cases aren't just imperfect, but irresponsible and unscientific with profound impacts on human lives. And that's the mission of this podcast to investigate how and why forensic methods are flawed and how the system can be fixed. From School of Humans, Curiosity Stream and iHeart Podcasts, this is CSI on Trial. Two or three stays are really not enough to call something an impacts patter from gunshot that's going to put someone in prison the rest of their life. Sto making up a lie was gonna get you home center. What is it about a bite mark that would make a dentist an expert in this area who shot at you? He said, I will sit in this jail and I will rot before I take the problem with forensic science in the criminal legal system today is that it's an awful lot of forensic and not an awful lot of science. Episode one, the Flaws of Forensic Science. Popular television has made forensic science famous. True crime has been a long time fascination for the American public, from the Salem witch trials to the Missing Lindbergh Baby, to the prosecution of gangsters like al Capone. Al Capone goes to jail, the federal government strikes its first telling law at the underworld by convicting public Enemy number one on income tax brauds. Modern forensic science launches because of a massive influx of federal funding in the late nineteen sixties, after PubL unrest during the Civil Rights Movement and protests against the Vietnam War spur President Lyndon Johnson to declare a war on crime to attack and to overcome growing crime and lawlessness. I think we must have a stepped up program to help modernize and strengthen our local police forces. Our people have a right to feel secure in their homes and on their streets, and that right just must be secured. It marks the first moment in American history where the federal government becomes directly involved in local law enforcement. Menka Sinha spent a decade as a public defender in Washington, d c. She's now a law professor and teaches the Criminal Defense Clinic at the University of Maryland School of Law. She's published on how forensic science can contribute to wrongful convictions. And they believe that this crime problem originates in these black urban communities, and without really consulting with the residents of these communities themselves, they do this investigation and they do the study, and they produced this really sweeping report that was issued in nineteen sixty seven. President Johnson put together a crime Commission with a mix of backgrounds in law education, social work, and law enforcement. The report they issued was called The Challenge of Crime in a Free Society. It had over two hundred recommendations, one of them the creation of a national emergency number, you know it as nine one. One. One of the things the commission points to is it says we need to improve our law enforcement technique by improving our capability of analyzing evidence left at scenes. So we should be improving our ability to do fingerprint nails, we should be improving our ability to do shoeprint analysis, and it mentioned the whole bunch of them, and it says we should be developing a centralized fingerprint database. And this recommendation becomes the backbone of the development of crime labs. Using the Commission's report and energized by the political climate of the time, Congress creates a major federal funding pipeline for state and local police forces, including funding for forensic science development and training. In order for police to be eligible for many of the grants, they have to show an increase in arrest rates. In other words, the development of modern forensic science was tied to and incentivized by arrests and prosecutions. Notice what I didn't say fueled modern forensic science, and that's science, independent, rigorous science. Here's Manca again. We tend to think of forensics as really reliable, really accurate, really scientific, And what most people don't realize is that most forensic methods they weren't developed in scientific labs. They weren't developed through scientific method They weren't developed through scientific research. They were developed by and for law enforcement to help law enforcement pursue law enforcement functions, meeting surveillance, investigation, prosecution, securing convictions, and sometimes even at sentencing to secure a higher punishment. The problem with forensic science in the criminal legal system today is that it's an awful lot of forensic and not an awful lot of science. That's Kate Judson. I'm the executive director of the Center for Integrity and Forensic Sciences. Much of it was developed for law enforcement for use in courtrooms rather than at a lab bench in academia, for example, and for that reason attend to be quite skewed. It tends to be in pursuit of a results or an agenda, rather than simply going where the science leads. In the late nineteen eighties, a proverbial bomb dropped on forensic sciences. DNA analysis developed by the scientific community far away from the world of arrests and prosecutions. DNA analysis held up a mirror to forensic science methods. My name's Julia Layton. I'm the former General Counsel for the DC Public Defender Service, but I've been retired for several years now. When I went to law school, most of the forensic disciplines that had been admitted for decades. So in the nineteen eighties when I started practicing, we weren't challenging the evidence. We weren't even looking very closely at the evidence. That sort of changed with the introduction of DNA forensic evidence, because suddenly this was something new and our responsibility as lawyers was to investigate it, learn about it, and so slowly but surely we started to learn about the technology behind it. We started to learn about the scientific method, and I think once we had started to master those topics, we naturally started to turn to look at the other disciplines. In contrast to DNA analysis, common forensic science disciplines firearms analysis, footprint comparison, byte mark analysis were revealed as unscientific. But most troubling and most revealing, DNA analysis was directly contradicting the validity of forensic evidence that had locked people in prison or even put them to death. In the early nineties, DNA technology becomes ripe enough for us to use in criminal cases. We can go back and test samples that have never been tested what they learned was that actually, we have convicted a lot of innocent people, and one of the major contributing factors they found was some sort of shoddy forensic evidence that was used in the first place. So everyone saw what was happening with DNA and exonerations and there was a huge change in the way we developed and used forensic science. Right. Well, no, it turns out there were more powerful forces than science and truth at play. It was a very impressive set of expertise in this committee and their report was really damning, and so for us it was like finally somebody's opened the door, opened Pandora's box about how little research, how little foundational, empirical, scientifically valid research has been done, and we thought we were going to go forth and have a sea change, and we were wrong. When DNA analysis began proving the innocence of people originally convicted on the weight of forensics science, Congress did act in the late two thousands. It funded a commission to study the issue and tapped the National Academy of Sciences to run it. The commission released its report in two thousand and nine. It was titled Strengthening Forensic Science in the United States a path forward. Manka Sinha says, the reports authors made sure it had a tone of optimism rather than blame. So the report's goal was really to improve the forensic system. So it makes all these recommendations for how to do that. We're gonna come up with a research strategy and guide how research should be funded. We're going to work on standards and regulations to regulate the use of forensics, and we're going to all do that centrally and through this national governing body. David Fegman is the Chancellor and dean at UC Hastings College of Law in San Francisco. So the report was published in two thousand and nine, and one of the things that the report advocated was for a federal agency to oversee the development of forensic science that would be able to accredit and certify labs, would oversee research and fund research, not unlike the National Institutes of Health might do. I'll just say it, this report came out in two thousand and nine. It's many many years after that, and there's no national governing forensic body. It never happened, It never got off the ground. Forensic analysts and their industry associations made some cosmetic changes to the language they used. The Department of Justice continued to provide research money, and some of the money did go to actual scientists. Some forensic associations and labs required more training hours, assessments, or accreditations, but there were no fundamental changes to what techniques were used. To be fair, the report was released during a tumultuous time. The research was done under the Bush administration, but released a few months after President Obama took office. The global economic crash sucked up all of the oxygen in US public policy, But years later, the Obama administration decided to pick up where the Congressional study left off. The President's Council of Advisors on Science and Technology, also known as PEACAST, pulled together the nation's top scientific talent to dig deeper into specific forensic science areas. University of Maryland physics professor Jim Gates was a principal author. Science is all about measurement, So the PEACASH report said, we are going to look at parts of forensic science that are based on measurement and ask the question, when we look at processes and practices in forensic science, how closely do those adhere to measurements made outside of forensic science, and unfortunately not much improvement had occurred. Professor Thomas Albright from the Salk Institute also contributed to the report, which came out in twenty sixteen. The Peacast report broadly condemned a particular category of forensic practices known as the feature comparison techniques. These are kenner fingerprints, tire tracks, bulletmarks, and so forth, and the major concern was that they hadn't been in most cases validated properly validated. Validation simply means that you've empirically tested the technique so that you're certain that a large fraction of the time it gives you the correct answer. But the Department of Justice, which has been entwined with forensic science since lbj's War on Crime agenda, rejected the findings on the day of its release. Attorney General Lauretta Lynch, who was the Attorney General at the time, she says, no, thank you, Peacast. We appreciate your efforts, but we are not going to adopt your recommendations. Attorney General Lynch said in a statement reported at the time by The Wall Street Journal that quote, we remain confident that when used properly, forensic science evidence helps juries identify the guilty and clear the innocent, and the Department believes that the current legal standards regarding the admissibility of forensic evidence are based on sound science and sound legal reasoning. The National District Attorney's Association they denounced the report immediately. The FBI criticizes the report. A whole bunch of bodies criticize the report and decline to adopt the recommendations. William Fitzpatrick from the National District Attorney's Association made it pretty clear how local prosecutors like himself felt if I were to have taken the Peacast reports seriously, which I did, or the two thousand and nine NAS report, which I did not, as well, and to embrace some recommendations suggested, I could actually picture myself in my office telling a grieving father of the domestic violence victim that even though his daughter's ex boyfriend was ideed fleeing the scene, even though he left a fingerprint that matched his left thumb in the victim's blood at the scene, and even though a friends at godentologists matched a bite mark on the victim's chest to the defendant and the thirty two caliber revolver recovered from his jacket pocket was ballistically matched to the projectile and his daughter's head. It's just right now, the case is too thin to proceed and I won't be able to prosecute please. They were worried that what it would mean for these closed cases, would they open back up? Would we have to relitigate all of this? And they were also worried about what it meant for open cases, even though the authors of the Peacast report were careful to say they had no view on the legal question of whether any past cases were erroneously decided. People convicted of a crime where questionable forensic science was used might be able to appeal their case and even walk free. Professor Jim Gates, again one of the report's authors, there's a large part of the pushback that comes from attorneys, quite frankly, particularly prosecuting attorneys. And one can understand why are there motivations for this, because in the current state, a forensic science is a tool that basically they can use to get convictions. The only problem, as I said, as an outside scientist, many of us outside the field say, this isn't science. You're using the power science. But you're not doing science. And this is a deep moral as well as practical problem. I have no criticism at all of the people who actually do the forensic science. It's a systemic problem that exists, and that systemic problem impact some communities more than others. Black people and brown people and other people for marginalized communities, but especially black people, suffer the punitive effects of our criminal legal system more and more harshly at every single stage of the process. A recent report by the National Registry of Exonerations found that Black Americans are seven times more likely to be wrongly convicted of serious crimes than white Americans. So, if that's the snapshot, and then you have a prosecutor or a prosecutorial entity saying, well, I won't be able to get a conviction if I don't have this tool. Well, if you particularly want to take somebody's liberty away, who is more likely to be black or brown? In this context, we should be making sure it's reliable and producing reliable results. At the least, if you should be able to do any of this at all, you should be doing it with tools that are reliable. Both prosecutors and defense attorneys call on expert witnesses to enter forensic science evidence into court and to explain that evidence to the jury. Judges are supposed to be the gatekeepers of those expert witnesses to make sure junk science or junk scientists aren't allowed in court. But that's not happening. Sometimes you have experts who come in and lab codes and it feels really scientific, even when it's not. It can have this effect of making other aspects of a case that are questionable sort of wash them away. We've talked about how law enforcement and prosecutors rely on forensic science, but what about judges. How do they fit in a critical Supreme Court ruling in nineteen ninety three set as standard on evidence judges are supposed to follow. Here's David Fegman again. He specializes in the use of scientific research and legal decision making. And then you had in nineteen ninety three, the United States Supreme Court decided what is now a landmark case, Daubert versus Merldal Pharmaceuticals. And what the US Supreme Court said was that judges under the federal rules of evidence had to be gatekeepers and they had to look at the underlying methods and principles for scientific evidence that's being offered in court. It's called the Daubert standard after the court case that established it. So what happened in the early nineties is courts started look at what the data actually were, what the methods that were being used, and they realized that there wasn't much there. And a good example of that is handwriting in the handwriting area. Quote started asking well, where the data, and the forensic scientists basically said, well, we don't have data, we don't have studies, but we know we can do it because we've been doing it for twenty five years. So trust us. And judges said, well, under the Daubert rule, you have to tell us the statistics. You have to give us the research studies. They have to be pure viewed in mainstream scientific journals. And these experts started saying, we don't have any of that, and judges, I think we're surprised. Through pre trial evidentiary hearings, judges are supposed to evaluate the forensic science and rule on whether or not a jury should hear it. What people forget is that judges are human being subject to the same kind of cognitive pressures that other people are This is Nancy Gertner. I was a criminal defense lawyer and a civil rights lawyer for twenty four years, and I was on the bench federal bench for seventeen and now I teach, I write, I do some litigation. I'm sort of a gadfly in lots of areas. When judges are asked to decide on these issues, they often fall back on established precedent. If forensic evidence has been admitted over the years have in an unchallenged way, then there's precedent for it, and a judge, a trial judge, knows that allowing it in is consistent with the precedent. It enforces this feedback loop that keeps us from critically examining forensic evidence. Overruling that precedent is rare because it's a big deal. A ruling like that is like a rock thrown in a pond. It can send ripples system wide. So judges will often let the evidence and the expert witness in and then the chances are that if you let in evidence like this, you'll be affirmed. You will be affirmed on appeal, and you'll be affirmed on appeal. Because the review of evidentiary decisions that a judge makes is actually quite cursory, is really not very substantial. So if a judge says, I'm going to allow in this arson evidence based on the precedent, even if it's unexamined precedent over years and years and years and years, the odds are you'll be affirmed. And I think that that kept lawyers from raising the issues. It's good for the system to have standards of some threshold reliability and the irony or the tragedy is on the civil side, the courts are pretty good at restricting what comes in. And it's really been on the criminal side and the forensic side that it's been a much steeper held a quie. So that means on the criminal side juries not judges are often weighing forensic evidence. The judge, you'll say, well, I'll err on the side of let the jury decide that way. It won't be error that way. I won't be making a mistake that way. Know, the acquittal in this case of the hung jury won't be on me. So if you're a juror and you're sitting in a courtroom, you are likely to weigh evidence that you perceive to be scientific or from an expert more highly than other pieces of evidence. In that case. This is Manka Sinha again, and that's a fundamental problem because the jurors don't know about the carcerel origins of these disciplines. Jurors don't know about the criticism and the scrutiny that these disciplines have received for failing to really shore up the scientific underpinnings. Jurors don't know that stuff. Scientific studies have revealed that forensic science testimony has outsized influence with juries. Jay Kohler teaches at Northwestern's Pritzker School of Law and studies behavioral decision theory. Studies show that jurors generally do give a lot of weight to expert testimony, and because this testimony is often technical in nature, jurors tend to rely on the experts credentials and experience when deciding how much weight to give to this testimony. But some studies show that jurors don't particularly care about whether the methods used by the experts have been scientifically tested or not. Instead, they're generally content to trust the credential and experienced expert. This is a problem in some forensic sciences, where most experts are trained and experienced, but where the methods used haven't been subjected to rigorous scientific testing. When these experts testify in trials, they tend to use one phrase over and over, reasonable scientific certainty. But what does that even mean? To a reasonable degree of scientific certainty? Is kind of a bogus phrase. It doesn't really have any meaning. It's just a phrase that has caught on in the courtrooms over the last ten or twenty years. Unfortunately, now people are onto it and realizing that's just a misleading wave of saying. In my opinion, here's what I think has happened. The phrase came about in nineteen thirty five as a ploy from a lawyer. The case was about a death resulting from a capsized boat. The plaintiff's lawyer asked a witness, a man who had experienced building this kind of motor boat, if he could quote determine with reasonable scientific certainty the cause of the capsizing of the boat. So a guy who had experience in building boats, not a scientist, was providing scientific certainty. David Fegman, most of the folks that testify in court for prosecutors are not scientists themselves. They are technicians. They are simply applying a technology that has been applied for decades, if not a century. And if you keep applying the same protocols and the protocols are not producing valid, good information, then you just keep on doing it over and over again, and there's no real testing of it. So let's recap the state of forensic science. Many of the methods were all familiar with from popular TV shows like CSI are used in crime labs and presented in courtrooms all across the country without ever having been subject to scientific validation. That means there is no proof the methods work. And then DNA comes along and starts proving that some people who you'll hear from in this podcast whose convictions were based on CSI evidence are actually innocent. The wider consequences of those exonerations is, of course, that the CSI evidence therefore had to be wrong. There have been at least two major national studies calling for independent oversight of forensic science outside the criminal legal system. That oversight hasn't happened. Meanwhile, the Supreme Court says judges are supposed to play a key role in keeping bad science out of the courtroom, but many don't want to make waves. So instead, juries are often left to decide, and juries find scientific experts to be very convincing and influential. Sometimes, when my kids were little, we'd resolve a conflict in family court. Each kid could make their case and then the judge me would rule, and everyone had to agree to abide by the ruling. I think I mostly got it right, but even if someone felt wronged, it was better than fighting it out in the backyard, and hopefully they knew I was trying. That's sort of how I thought the justice system worked. Sure, a few eggs might get broken, but generally it's better than the alternative. But no amount of fair mindedness really matters. If the basic tools a decision is based on our flawed. So if the s in CSI isn't verified science, where does that leave things? How should we view forensic science. Here's what former public defender Julia Layton thinks. Forensics is a high risk industry. It's no different than medicine and aviation. People's lives are at stake, public safety is at stake. Getting it right matters, and we should be treating it as a high risk industry. We should be regulating it. We should be investing in it and we should be overseeing it. So think about that. Think about sitting on a plane with your family about to take off for a long flight. You might be afraid of flying, but logic tells you it's safe because the FAA, the air traffic controllers, the airline, the pilot, and the crew have all checked the data. The plane couldn't even be in service if it wasn't safe. Think about picking up your medication from the pharmacist. Do you pause before swallowing a pill to consider whether or not the pharmaceutical company was on the ball when they developed it, or do you assume it's safe because the industry is regulated. Now, what's being done to protect you or a loved one from junk forensic science? We now have four hundred public crime labs operating across the US, along with countless private contractors. In the coming episodes of this podcast, we will dive into specific methods like bite marks, bloodstained pattern analysis, and footprints. We'll explain the history behind the method, how it's practiced, and how reliable it really is. Kate Judson wants us to keep in mind that most people that walk into a courtroom are doing it with the best of intentions to find justice for a victim. I think that the majority of people who contribute to bad science coming into the courtroom are doing so because they are really trying to be right. I mean, they want the right result. They want to make sure that yield to people go to prison and in US and people go free. And they just are working within a system that doesn't have the quality assurances it should have. On the next episode of CSI on Trial, what is it about a bite mark that would make a dentist an expert in this area? Why use bite mark evidence when just this year long to other people besides myself have gotten out and bite mark evidence was used in the trial. I mean, explain to me what does it take to realize this stuff's all crap. CSI on Trial is a coproduction of iHeart Podcasts and School of Humans based on the Curiosity Stream series CSI on Trial, created by Eleanor Grant and produced by The Biscuit Factory. You can watch all six episodes of the video series right now at curiosity stream dot com. This episode is hosted and written by me Molly Herman and researched by Katie Dunn and myself. Our producer is Miranda Hawkins. Jessica Metzger is the senior producer. Virginia Prescott, Jason English, Brandon Barr and L. C. Crowley are the executive producers. Sound design in mix by Miranda Hawkins. Special thanks to John Higgins, Rob Burke, Rob Lyle and Brandon Craigie. If you're enjoying the show, leave a review in your favorite podcast app. Check out the Curiosity Audio Network for podcasts covering history, pop culture, true crime, and more. School of Humans

Oh can you name this TV show? So the width of the burn and the pinpoint detail in the fuel, I've narrowed down what our suspects using. Burned the tags off a blowtworks. Yeah, burns a small precise, just like what you get with compact torches used in dental labs. It's CSI Crime Scene Investigation, a CBS crime drama featuring a team of investigators solving crimes by analyzing the physical evidence. See once a dried, the pattern revealed itself. You see this discoloration right here. Looks at the tread from a boot well I initially thought was a V is actually a WU And that's the logo for Woodridge Hunting boots. They look at things like bloodstain patterns, footprints, and bullet casings. The original show launched in two thousand, then came the spin and offs, inspiring still more shows that dig into forensic science, Bones and CIS Dexter. The list is long. These crime scene procedurals. They're fun, they make investigations digestible, engaging millions watch them. But these shows also propel the idea that forensic science is precise in proving guilt or innocence. It's called the CSI effect. The CSI effect, from a defense attorney's perspective is is that and shows like that they depict forensic evidence as infallible, and that it's quick, it's fast, and they do it in a half an hour, and they wrap it all up right, and that jurors come into lajers come into a criminal case with the expectation that forensic evidence can conceivably do that. That's Chris fabricand I'm the director of Strategic Litigation at the Innocence Project in New York, and i am the author of Junk Science and the American Criminal Justice System. CSI and shows like it give jurors a false sent of security that forensic science methods are totally accurate, that expert testimony from CSI analysts is infallible. Studies show that juries put an outsized amount of trust in the testimony of experts. But as we'll discuss in this podcast, forensic science methods and the expert testimony that shares that evidence with juries can both be flawed. It's an issue that has serious consequences. CSI this all crap. That's Keith Harward, one of several people you'll meet in this series, who was wrongly convicted based on forensic evidence that turned out to be wrong. I'm Molly Herman. I'm a filmmaker and producer of a six part documentary series for Curiosity Stream that digs into forensic crime science, one discipline at a time. I've interviewed nearly seventy practitioners, attorneys, scientists, journalists, judges, and innocent people who spend decades in prison, some who are still there. When I started this project, I believed that old adage that the US justice system isn't perfect, but it's still the best system out there. But what I discovered is that many methods of forensic science used to build criminal cases aren't just imperfect, but irresponsible and unscientific with profound impacts on human lives. And that's the mission of this podcast to investigate how and why forensic methods are flawed and how the system can be fixed. From School of Humans, Curiosity Stream and iHeart Podcasts, this is CSI on Trial. Two or three stays are really not enough to call something an impacts patter from gunshot that's going to put someone in prison the rest of their life. Sto making up a lie was gonna get you home center. What is it about a bite mark that would make a dentist an expert in this area who shot at you? He said, I will sit in this jail and I will rot before I take the problem with forensic science in the criminal legal system today is that it's an awful lot of forensic and not an awful lot of science. Episode one, the Flaws of Forensic Science. Popular television has made forensic science famous. True crime has been a long time fascination for the American public, from the Salem witch trials to the Missing Lindbergh Baby, to the prosecution of gangsters like al Capone. Al Capone goes to jail, the federal government strikes its first telling law at the underworld by convicting public Enemy number one on income tax brauds. Modern forensic science launches because of a massive influx of federal funding in the late nineteen sixties, after PubL unrest during the Civil Rights Movement and protests against the Vietnam War spur President Lyndon Johnson to declare a war on crime to attack and to overcome growing crime and lawlessness. I think we must have a stepped up program to help modernize and strengthen our local police forces. Our people have a right to feel secure in their homes and on their streets, and that right just must be secured. It marks the first moment in American history where the federal government becomes directly involved in local law enforcement. Menka Sinha spent a decade as a public defender in Washington, d c. She's now a law professor and teaches the Criminal Defense Clinic at the University of Maryland School of Law. She's published on how forensic science can contribute to wrongful convictions. And they believe that this crime problem originates in these black urban communities, and without really consulting with the residents of these communities themselves, they do this investigation and they do the study, and they produced this really sweeping report that was issued in nineteen sixty seven. President Johnson put together a crime Commission with a mix of backgrounds in law education, social work, and law enforcement. The report they issued was called The Challenge of Crime in a Free Society. It had over two hundred recommendations, one of them the creation of a national emergency number, you know it as nine one. One. One of the things the commission points to is it says we need to improve our law enforcement technique by improving our capability of analyzing evidence left at scenes. So we should be improving our ability to do fingerprint nails, we should be improving our ability to do shoeprint analysis, and it mentioned the whole bunch of them, and it says we should be developing a centralized fingerprint database. And this recommendation becomes the backbone of the development of crime labs. Using the Commission's report and energized by the political climate of the time, Congress creates a major federal funding pipeline for state and local police forces, including funding for forensic science development and training. In order for police to be eligible for many of the grants, they have to show an increase in arrest rates. In other words, the development of modern forensic science was tied to and incentivized by arrests and prosecutions. Notice what I didn't say fueled modern forensic science, and that's science, independent, rigorous science. Here's Manca again. We tend to think of forensics as really reliable, really accurate, really scientific, And what most people don't realize is that most forensic methods they weren't developed in scientific labs. They weren't developed through scientific method They weren't developed through scientific research. They were developed by and for law enforcement to help law enforcement pursue law enforcement functions, meeting surveillance, investigation, prosecution, securing convictions, and sometimes even at sentencing to secure a higher punishment. The problem with forensic science in the criminal legal system today is that it's an awful lot of forensic and not an awful lot of science. That's Kate Judson. I'm the executive director of the Center for Integrity and Forensic Sciences. Much of it was developed for law enforcement for use in courtrooms rather than at a lab bench in academia, for example, and for that reason attend to be quite skewed. It tends to be in pursuit of a results or an agenda, rather than simply going where the science leads. In the late nineteen eighties, a proverbial bomb dropped on forensic sciences. DNA analysis developed by the scientific community far away from the world of arrests and prosecutions. DNA analysis held up a mirror to forensic science methods. My name's Julia Layton. I'm the former General Counsel for the DC Public Defender Service, but I've been retired for several years now. When I went to law school, most of the forensic disciplines that had been admitted for decades. So in the nineteen eighties when I started practicing, we weren't challenging the evidence. We weren't even looking very closely at the evidence. That sort of changed with the introduction of DNA forensic evidence, because suddenly this was something new and our responsibility as lawyers was to investigate it, learn about it, and so slowly but surely we started to learn about the technology behind it. We started to learn about the scientific method, and I think once we had started to master those topics, we naturally started to turn to look at the other disciplines. In contrast to DNA analysis, common forensic science disciplines firearms analysis, footprint comparison, byte mark analysis were revealed as unscientific. But most troubling and most revealing, DNA analysis was directly contradicting the validity of forensic evidence that had locked people in prison or even put them to death. In the early nineties, DNA technology becomes ripe enough for us to use in criminal cases. We can go back and test samples that have never been tested what they learned was that actually, we have convicted a lot of innocent people, and one of the major contributing factors they found was some sort of shoddy forensic evidence that was used in the first place. So everyone saw what was happening with DNA and exonerations and there was a huge change in the way we developed and used forensic science. Right. Well, no, it turns out there were more powerful forces than science and truth at play. It was a very impressive set of expertise in this committee and their report was really damning, and so for us it was like finally somebody's opened the door, opened Pandora's box about how little research, how little foundational, empirical, scientifically valid research has been done, and we thought we were going to go forth and have a sea change, and we were wrong. When DNA analysis began proving the innocence of people originally convicted on the weight of forensics science, Congress did act in the late two thousands. It funded a commission to study the issue and tapped the National Academy of Sciences to run it. The commission released its report in two thousand and nine. It was titled Strengthening Forensic Science in the United States a path forward. Manka Sinha says, the reports authors made sure it had a tone of optimism rather than blame. So the report's goal was really to improve the forensic system. So it makes all these recommendations for how to do that. We're gonna come up with a research strategy and guide how research should be funded. We're going to work on standards and regulations to regulate the use of forensics, and we're going to all do that centrally and through this national governing body. David Fegman is the Chancellor and dean at UC Hastings College of Law in San Francisco. So the report was published in two thousand and nine, and one of the things that the report advocated was for a federal agency to oversee the development of forensic science that would be able to accredit and certify labs, would oversee research and fund research, not unlike the National Institutes of Health might do. I'll just say it, this report came out in two thousand and nine. It's many many years after that, and there's no national governing forensic body. It never happened, It never got off the ground. Forensic analysts and their industry associations made some cosmetic changes to the language they used. The Department of Justice continued to provide research money, and some of the money did go to actual scientists. Some forensic associations and labs required more training hours, assessments, or accreditations, but there were no fundamental changes to what techniques were used. To be fair, the report was released during a tumultuous time. The research was done under the Bush administration, but released a few months after President Obama took office. The global economic crash sucked up all of the oxygen in US public policy, But years later, the Obama administration decided to pick up where the Congressional study left off. The President's Council of Advisors on Science and Technology, also known as PEACAST, pulled together the nation's top scientific talent to dig deeper into specific forensic science areas. University of Maryland physics professor Jim Gates was a principal author. Science is all about measurement, So the PEACASH report said, we are going to look at parts of forensic science that are based on measurement and ask the question, when we look at processes and practices in forensic science, how closely do those adhere to measurements made outside of forensic science, and unfortunately not much improvement had occurred. Professor Thomas Albright from the Salk Institute also contributed to the report, which came out in twenty sixteen. The Peacast report broadly condemned a particular category of forensic practices known as the feature comparison techniques. These are kenner fingerprints, tire tracks, bulletmarks, and so forth, and the major concern was that they hadn't been in most cases validated properly validated. Validation simply means that you've empirically tested the technique so that you're certain that a large fraction of the time it gives you the correct answer. But the Department of Justice, which has been entwined with forensic science since lbj's War on Crime agenda, rejected the findings on the day of its release. Attorney General Lauretta Lynch, who was the Attorney General at the time, she says, no, thank you, Peacast. We appreciate your efforts, but we are not going to adopt your recommendations. Attorney General Lynch said in a statement reported at the time by The Wall Street Journal that quote, we remain confident that when used properly, forensic science evidence helps juries identify the guilty and clear the innocent, and the Department believes that the current legal standards regarding the admissibility of forensic evidence are based on sound science and sound legal reasoning. The National District Attorney's Association they denounced the report immediately. The FBI criticizes the report. A whole bunch of bodies criticize the report and decline to adopt the recommendations. William Fitzpatrick from the National District Attorney's Association made it pretty clear how local prosecutors like himself felt if I were to have taken the Peacast reports seriously, which I did, or the two thousand and nine NAS report, which I did not, as well, and to embrace some recommendations suggested, I could actually picture myself in my office telling a grieving father of the domestic violence victim that even though his daughter's ex boyfriend was ideed fleeing the scene, even though he left a fingerprint that matched his left thumb in the victim's blood at the scene, and even though a friends at godentologists matched a bite mark on the victim's chest to the defendant and the thirty two caliber revolver recovered from his jacket pocket was ballistically matched to the projectile and his daughter's head. It's just right now, the case is too thin to proceed and I won't be able to prosecute please. They were worried that what it would mean for these closed cases, would they open back up? Would we have to relitigate all of this? And they were also worried about what it meant for open cases, even though the authors of the Peacast report were careful to say they had no view on the legal question of whether any past cases were erroneously decided. People convicted of a crime where questionable forensic science was used might be able to appeal their case and even walk free. Professor Jim Gates, again one of the report's authors, there's a large part of the pushback that comes from attorneys, quite frankly, particularly prosecuting attorneys. And one can understand why are there motivations for this, because in the current state, a forensic science is a tool that basically they can use to get convictions. The only problem, as I said, as an outside scientist, many of us outside the field say, this isn't science. You're using the power science. But you're not doing science. And this is a deep moral as well as practical problem. I have no criticism at all of the people who actually do the forensic science. It's a systemic problem that exists, and that systemic problem impact some communities more than others. Black people and brown people and other people for marginalized communities, but especially black people, suffer the punitive effects of our criminal legal system more and more harshly at every single stage of the process. A recent report by the National Registry of Exonerations found that Black Americans are seven times more likely to be wrongly convicted of serious crimes than white Americans. So, if that's the snapshot, and then you have a prosecutor or a prosecutorial entity saying, well, I won't be able to get a conviction if I don't have this tool. Well, if you particularly want to take somebody's liberty away, who is more likely to be black or brown? In this context, we should be making sure it's reliable and producing reliable results. At the least, if you should be able to do any of this at all, you should be doing it with tools that are reliable. Both prosecutors and defense attorneys call on expert witnesses to enter forensic science evidence into court and to explain that evidence to the jury. Judges are supposed to be the gatekeepers of those expert witnesses to make sure junk science or junk scientists aren't allowed in court. But that's not happening. Sometimes you have experts who come in and lab codes and it feels really scientific, even when it's not. It can have this effect of making other aspects of a case that are questionable sort of wash them away. We've talked about how law enforcement and prosecutors rely on forensic science, but what about judges. How do they fit in a critical Supreme Court ruling in nineteen ninety three set as standard on evidence judges are supposed to follow. Here's David Fegman again. He specializes in the use of scientific research and legal decision making. And then you had in nineteen ninety three, the United States Supreme Court decided what is now a landmark case, Daubert versus Merldal Pharmaceuticals. And what the US Supreme Court said was that judges under the federal rules of evidence had to be gatekeepers and they had to look at the underlying methods and principles for scientific evidence that's being offered in court. It's called the Daubert standard after the court case that established it. So what happened in the early nineties is courts started look at what the data actually were, what the methods that were being used, and they realized that there wasn't much there. And a good example of that is handwriting in the handwriting area. Quote started asking well, where the data, and the forensic scientists basically said, well, we don't have data, we don't have studies, but we know we can do it because we've been doing it for twenty five years. So trust us. And judges said, well, under the Daubert rule, you have to tell us the statistics. You have to give us the research studies. They have to be pure viewed in mainstream scientific journals. And these experts started saying, we don't have any of that, and judges, I think we're surprised. Through pre trial evidentiary hearings, judges are supposed to evaluate the forensic science and rule on whether or not a jury should hear it. What people forget is that judges are human being subject to the same kind of cognitive pressures that other people are This is Nancy Gertner. I was a criminal defense lawyer and a civil rights lawyer for twenty four years, and I was on the bench federal bench for seventeen and now I teach, I write, I do some litigation. I'm sort of a gadfly in lots of areas. When judges are asked to decide on these issues, they often fall back on established precedent. If forensic evidence has been admitted over the years have in an unchallenged way, then there's precedent for it, and a judge, a trial judge, knows that allowing it in is consistent with the precedent. It enforces this feedback loop that keeps us from critically examining forensic evidence. Overruling that precedent is rare because it's a big deal. A ruling like that is like a rock thrown in a pond. It can send ripples system wide. So judges will often let the evidence and the expert witness in and then the chances are that if you let in evidence like this, you'll be affirmed. You will be affirmed on appeal, and you'll be affirmed on appeal. Because the review of evidentiary decisions that a judge makes is actually quite cursory, is really not very substantial. So if a judge says, I'm going to allow in this arson evidence based on the precedent, even if it's unexamined precedent over years and years and years and years, the odds are you'll be affirmed. And I think that that kept lawyers from raising the issues. It's good for the system to have standards of some threshold reliability and the irony or the tragedy is on the civil side, the courts are pretty good at restricting what comes in. And it's really been on the criminal side and the forensic side that it's been a much steeper held a quie. So that means on the criminal side juries not judges are often weighing forensic evidence. The judge, you'll say, well, I'll err on the side of let the jury decide that way. It won't be error that way. I won't be making a mistake that way. Know, the acquittal in this case of the hung jury won't be on me. So if you're a juror and you're sitting in a courtroom, you are likely to weigh evidence that you perceive to be scientific or from an expert more highly than other pieces of evidence. In that case. This is Manka Sinha again, and that's a fundamental problem because the jurors don't know about the carcerel origins of these disciplines. Jurors don't know about the criticism and the scrutiny that these disciplines have received for failing to really shore up the scientific underpinnings. Jurors don't know that stuff. Scientific studies have revealed that forensic science testimony has outsized influence with juries. Jay Kohler teaches at Northwestern's Pritzker School of Law and studies behavioral decision theory. Studies show that jurors generally do give a lot of weight to expert testimony, and because this testimony is often technical in nature, jurors tend to rely on the experts credentials and experience when deciding how much weight to give to this testimony. But some studies show that jurors don't particularly care about whether the methods used by the experts have been scientifically tested or not. Instead, they're generally content to trust the credential and experienced expert. This is a problem in some forensic sciences, where most experts are trained and experienced, but where the methods used haven't been subjected to rigorous scientific testing. When these experts testify in trials, they tend to use one phrase over and over, reasonable scientific certainty. But what does that even mean? To a reasonable degree of scientific certainty? Is kind of a bogus phrase. It doesn't really have any meaning. It's just a phrase that has caught on in the courtrooms over the last ten or twenty years. Unfortunately, now people are onto it and realizing that's just a misleading wave of saying. In my opinion, here's what I think has happened. The phrase came about in nineteen thirty five as a ploy from a lawyer. The case was about a death resulting from a capsized boat. The plaintiff's lawyer asked a witness, a man who had experienced building this kind of motor boat, if he could quote determine with reasonable scientific certainty the cause of the capsizing of the boat. So a guy who had experience in building boats, not a scientist, was providing scientific certainty. David Fegman, most of the folks that testify in court for prosecutors are not scientists themselves. They are technicians. They are simply applying a technology that has been applied for decades, if not a century. And if you keep applying the same protocols and the protocols are not producing valid, good information, then you just keep on doing it over and over again, and there's no real testing of it. So let's recap the state of forensic science. Many of the methods were all familiar with from popular TV shows like CSI are used in crime labs and presented in courtrooms all across the country without ever having been subject to scientific validation. That means there is no proof the methods work. And then DNA comes along and starts proving that some people who you'll hear from in this podcast whose convictions were based on CSI evidence are actually innocent. The wider consequences of those exonerations is, of course, that the CSI evidence therefore had to be wrong. There have been at least two major national studies calling for independent oversight of forensic science outside the criminal legal system. That oversight hasn't happened. Meanwhile, the Supreme Court says judges are supposed to play a key role in keeping bad science out of the courtroom, but many don't want to make waves. So instead, juries are often left to decide, and juries find scientific experts to be very convincing and influential. Sometimes, when my kids were little, we'd resolve a conflict in family court. Each kid could make their case and then the judge me would rule, and everyone had to agree to abide by the ruling. I think I mostly got it right, but even if someone felt wronged, it was better than fighting it out in the backyard, and hopefully they knew I was trying. That's sort of how I thought the justice system worked. Sure, a few eggs might get broken, but generally it's better than the alternative. But no amount of fair mindedness really matters. If the basic tools a decision is based on our flawed. So if the s in CSI isn't verified science, where does that leave things? How should we view forensic science. Here's what former public defender Julia Layton thinks. Forensics is a high risk industry. It's no different than medicine and aviation. People's lives are at stake, public safety is at stake. Getting it right matters, and we should be treating it as a high risk industry. We should be regulating it. We should be investing in it and we should be overseeing it. So think about that. Think about sitting on a plane with your family about to take off for a long flight. You might be afraid of flying, but logic tells you it's safe because the FAA, the air traffic controllers, the airline, the pilot, and the crew have all checked the data. The plane couldn't even be in service if it wasn't safe. Think about picking up your medication from the pharmacist. Do you pause before swallowing a pill to consider whether or not the pharmaceutical company was on the ball when they developed it, or do you assume it's safe because the industry is regulated. Now, what's being done to protect you or a loved one from junk forensic science? We now have four hundred public crime labs operating across the US, along with countless private contractors. In the coming episodes of this podcast, we will dive into specific methods like bite marks, bloodstained pattern analysis, and footprints. We'll explain the history behind the method, how it's practiced, and how reliable it really is. Kate Judson wants us to keep in mind that most people that walk into a courtroom are doing it with the best of intentions to find justice for a victim. I think that the majority of people who contribute to bad science coming into the courtroom are doing so because they are really trying to be right. I mean, they want the right result. They want to make sure that yield to people go to prison and in US and people go free. And they just are working within a system that doesn't have the quality assurances it should have. On the next episode of CSI on Trial, what is it about a bite mark that would make a dentist an expert in this area? Why use bite mark evidence when just this year long to other people besides myself have gotten out and bite mark evidence was used in the trial. I mean, explain to me what does it take to realize this stuff's all crap. CSI on Trial is a coproduction of iHeart Podcasts and School of Humans based on the Curiosity Stream series CSI on Trial, created by Eleanor Grant and produced by The Biscuit Factory. You can watch all six episodes of the video series right now at curiosity stream dot com. This episode is hosted and written by me Molly Herman and researched by Katie Dunn and myself. Our producer is Miranda Hawkins. Jessica Metzger is the senior producer. Virginia Prescott, Jason English, Brandon Barr and L. C. Crowley are the executive producers. Sound design in mix by Miranda Hawkins. Special thanks to John Higgins, Rob Burke, Rob Lyle and Brandon Craigie. If you're enjoying the show, leave a review in your favorite podcast app. Check out the Curiosity Audio Network for podcasts covering history, pop culture, true crime, and more. School of Humans