Welcome to Stuff to Blow Your Mind from how Stop Work dot com. Hey, so, I just wanted to let you know that Christian and I ended up talking about this topic for a really long time. So we decided to split the episode in two, and this is going to be the first half of our discussion. You can tune in again next time to hear the second half. Welcome to Stuff to Blow Your Mind. My name is Joe McCormick and I'm Christian Sager. So our co host Robert is away this week. He's on vacation, and the two of us thought that this would be a good opportunity to talk about a little side project that we've been working on for like six months now, longer than that. Are you talking about that breeder reactor we've been working on? That's our side side project. This one is is a little bit more on the books. This is the one where we've been looking into d I Y for forensics. Oh yeah, so Christian and I, well may Christian, I want to give you all the credit because it's true that you've had this idea of wanting to produce a video series for How Stuff Works showing how you can do your own forensic science investigations just like you'd see Uh well, I don't want to say just like you'd see on c s I, because it turns out all all that stuff you see is mostly Faye garbage. Um, but like a forensic investigator would do it a crime scene, So so how you can use tools available to you to figure out what happened if say there's blood spatter all over a wall, or if there is a fingerprint left on a surface. And so the first one that we did was like a demo because we wanted to like kind of test it as a proof of concept, right, And we took one of our studios here at how Stuff works and we covered the walls with paper. Yeah, we got a mannequin head and put some fake blood packets on it. Yeah. And then um, we've basically had Lauren Vogel Baumb from Forward Thinking come in with Thor's hammer mulenir, which we just have laying around and just pound on this mannequin's head so that the blood would splatter as if it was a person being hit with a real hammer. It's to see how the blood was splatter. And we were testing basically the premise that you of how blood splatter analysis works and there's some math to it. So then we subsequently did the stringing, which they've probably seen on these forensic TV shows like C S I, where they string crime scenes and there you're trying to identify the trajectory of the blood that hit to leave the stained pattern that you find. Yeah, exactly, so you can sort of figure out both where the crime happened and what the like height. And I guess like three dimensionally is the best way to explain it, right, you can explain where it happened in the room, but also the height of the blow. Uh. And so we did it and it it worked, um sort of. Yeah, it was a mess with all of the complications we have. Ye. Yeah, I mean we definitely weren't as prepared for how difficult it was as I thought. And then we posted some photos to the stuff to blow your mind social media accounts, and at least one, maybe two people popped up if you're listening, thank you. Um they were blood spat spatter analysis experts, and they gave us some advice. Uh. And we still we've just been so busy with everything else that we do here, we just haven't been able to get to the rest of it. But the idea was we were going to take this and we're gonna extrapolate it out into like a four maybe five part series that was going to be a locked room murder mystery where each episode Joe and I tried to solve a crime by doing d I Y forensics in the room. And we were going to do the blood spatter one, We're going to do paper chromatography. We were talking about dusting for fingerprints with super glue, which is really interesting. We were going to talk about decomposition of bodies and how do you figure out the time of death. Then we got into looking at some other stuff. We had to bite marks, hair and fiber analysis, stuff like that, and we were like, how valid actually is this? I mean not not just beyond like us doing the I Y version of it with like ziplock, Baggies and tweezers here in our studios, but like, how actually valid is this? And Josh Clark from Stuff you Should Know started talking to us about it, and he was like, you know, there's a lot of really bad pseudo science in this, and he sent us a great article. This is something that Josh and I had talked about several times before. Actually it's sort of a running conversation we have about problems that keep emerging in forensic science. Absolutely. Yeah, So he pointed this out to us, and we thought, you know what, uh, why don't we do an episode of stuff to blow your mind about this was so we can sort of get our thoughts clear, really do a deep dive on the research, and then maybe one day we'll return to this video series and we'll make sure that the stuff that we're doing, first of all, is valid. But that second of all that we can what I'm hoping to do with it is also somehow within this locked room narrative that we're going to construct also tell the audience like you can't rely on hair evidence, for instance, right or bite marks because there's a lot of subjectivity to how that's done. Right now, So that's what we're here to talk to you about today, is the pseudoscience and the sort of false interpretations and there's there's just a lot of problems with forensic science, and I think people within the field that would acknowledge that as well. Yeah, and it certainly has been openly acknowledged. So one resource that we're gonna be referencing throughout this episode is this massive compilation research document put together by the National Research Council and published in two thousand nine called Strengthening Forensic Science in the United States. And this was put together out of a out of a commission I think funded by Congress to look at the state of forensic science in the United States and and take a scientific, analytical, critical approach to it to say how well are we using forensic science in our courts, Like how well how scientifically established are the methods that are being used, how often do they get the right answer as far as we can tell, Because it has become more and more clear that lots of traditional methods used in forensic science analysis, the kind of science we would use to analyze a crime scene, to identify a suspect, to demonstrate the guilt of a suspect, etcetera. Uh, these things are in many cases full of flaws. And I've seen it alleged that really nuclear DNA analysis is about the only forensic science discipline used in US courts that isn't thoroughly riddled with problems. Actually, and we'll talk about it in this episode two. There are even it can be vulnerable, but I think it's generally considered the best. It is considered the best, Yeah, but there are issues with it as well. UM So I got a question for you. Have you ever served on a jury before? No, I haven't. I've I've had you know, jury selection days, but I've never I've never been picked for a jury. So a couple of years ago, I was picked for a jury and I don't know how this happened, but somehow I ended up foreman and uh, little do they know what power they put in the hands of this uh, this doom purveyor. It was a real weird case and I won't take you all down the long road of it, but I will say you know, one of the people that came in it was a it was a cocaine uh possession and distribution case, but also a firearm possession case for felon who was not supposed to be carrying firearms. Um. The they had a cocaine expert come in from their forensics lab. And you know, like with any rhetorical position, especially in the courtroom, I mean this is where like a lot of the Greek terminology for for rhetoric came from. They first started off by presenting the ethos the quality of the character of this woman. She gave us her how many years that she had been working in the lab and in the field, and what her degrees were, and what her training was, and how many cases that she had looked at and all that stuff to sort of establish upfront, this is a person you should believe, right, And that's pretty much standard practice when you're in a courtroom and somebody who's a science expert comes or a forensics expert comes to testify about a case. But what you don't know is, necessarily, like the actual field itself, how much solidity there is to the discipline. Yeah. So imagine you are on a jury and it is eighteen thirty five or so, so you are not the scientifically literate person you are today. You are maybe a farmer who has been called in for jury duty, and it's a murder trial, and the prosecution brings on an expert witness to testify to the guilt of the defendant. And the expert witness who comes on says, look, here is a map of the different organs on the human skull. And as you will see in this sketch of the defendant's head, they have an extremely pronounced organ of murdering, and that's a bump that's right here on this part of the head, and the principles of phrenology, we can demonstrate that this person is by nature a murderer and will kill again if released. Now we know that that sounds like complete bunk. And wouldn't life be super easy if we could just touch everybody's heads and figure out if there are murderers? It would be much easier. It would also give you more excuses when you're touching the heads and people are saying, why are you doing? What do checking? Make sure? Make sure I'm safe. I don't want to be in the presence of somebody with a pronounced organ of destructiveness. Um but so, yeah, we all know today phrenology is nonsense. But back then a lot of people would not have had the knowledge to do that. And it's not because they were stupid, they just didn't know. It's probably presented in much the same way. This learned gentleman gets up in front of you and says, I am an expert, and all phrenologists agree. Um. So, in that case, if if the judge has decided that this is admissible testimony and you're on the jury, how would you know to quite usenet And then even if you're the judge, the judge might know not exactly. It's not a qualification to be a judge to also have a science degree exactly. Now we are going to talk, believe it or not, about a real case where somebody tried to use phrenology in the courtroom in the eighteen hundreds. But we'll get to that actual historical example in a bit. First we should back up, I think and and look a little bit more generally at a sort of top down view of what are some of the problems with forensic science as it's used, especially in U. S courts today, but this is going to apply to a lot of courts around the world in the general sense. Yeah, and and like as usual, because we're Americans and we're here in the US, a lot of our research is focused here. But the best resource that I was able to find came from Popular Mechanics, and they had a really good article on the myths of c s I and they began by establishing that forensic science as we know it it was mostly created by police, not by scientists, and it's based more on common sense rather than scientific practice. And we all know what common sense is. Common sense is the reasoning faculty that tells you that the Earth does not move and is flat. Yeah, b O B. I'm downe with B O B. Well, I mean those things are common whitest thing I've ever said on this show. I'm sorry, Uh yeah, I mean so common sense, as we know from all kinds of fields of science, very often betrays us. Common sense is useful for getting your average stuff done day today, but it is not good for for deducing truths that are obscure. Yeah. Right, So this popular mechanics article goes in depth and it says okay, and it and it points out DNA testing right off the bat, and they said, DNA testing is is really good, and it's made it possible for us to re examine all this other biological evidence that we've taken from past trials, so that more than two hundred people this is in the US have had their convictions overturned because the DNA analysis refutes the other biological evidence that was used in the case. Fifty percent of these cases involved bad forensic analysis contributing to their imprisonment, and they refer to this as the c s I affects of this bringing up the show. Yeah, the c s I effect, as I've kind of come to understand, let me know, if this is in line with what they're saying. As I understand it, it's where juries tend to expect to encounter scientific style evidence presented in the courts, or especially DNA evidence. Yeah. Absolutely, And so it's a perpetuation of the media versions of the forensic examiners solving difficult cases the science and cutting edge tech. Right, Like, so when you think of c s I and you think of, like, wow, this this is how old I am, Like David Caruso throwing on his sunglasses and whipping back his red hair, and then there's like c g I zooming in on a dead body, and like the explanation exposition about like how forensics works. That's what they're expecting, right, They're expecting some David Caruso slick type to show up and explain this to them. There's an argument though, that this is a misconception that absolutely sways jurors before they're really even in the courtroom. Yeah. Um, Like it leads to the misperception that generally the methods leading to the prosecution, identification and prosecution of a defendant are very scientific in nature. Not only that, but that they all cases will have some kind of scientific quote unquote scientific evidence, uh engaged or as evidence for them to review. Yeah. So, but what's actually going on is a little bit more difficult. Like like all things in the world, America's forensics labs are totally overburdened and understaffed, and a two thousand five studied by the Department of Justice found that the average lab has a backlog of four hundred and one requests. So that's why, Like, for instance, that case that I was the forem and on, I think it happened like a year year and a half after they actually arrested the guy because they were waiting on evidence. It took forever for them to actually get to the evidence into Like in this case, they were looking at the cocaine that was found on his person to see what percentage of it was pure cocaine. Um. So this isn't even getting into all of the state, city, and now federal forensic departments that have been scandalized by mishandling evidence or just straight up committing fraud. Uh So, I mean, even outside of the science angle that we're talking about. There are the pseudoscience angle we're talking about, there's the mishandling of evidence which makes the science bad, and then there's just people committing fraud, right, which is well, I mean unethical. I guess you could encounter fraud in any situation, like even in a even in a field where all of the established methodology is well vetted, scientifically valid methodology, you could still have somebody falsified data, somebody who has like the quote unquote quality of character to testify in a courtroom. But whatever, somebody somebody got to them. But or than just telling people not to be liars, there's really not as much to do about that. So I think we're probably not going to focus as much on fraud today, but it's just important to know that it's out there, right, especially like next time one of you is in the jury box. You know, just consider these things that we're talking about today. Uh So, the argument basically goes like this, the forensics science field, it has a foundation that's very shaky and very subjective. In fact, the American Society of Crime Laboratory Directors says, look, there's no advanced degree that's required to have a career in forensics. Some people do, but it's not inquired. I think it varies field to field because there will be for example, like maybe like boards or licensing organizations that work in different fields and they have different requirements. Sure yeah, especially probably based state to state all that stuff. Um, And one question I have and let's I don't want you. I mean, you can answer it right now or we can think about it and come back to it the end of the episode. But should it be should they be required to have an advanced degree? Uh? Well, I mean I think that's gonna very case to case again. I think I think methodology is more important than credential ing. Yeah yeah, Um, Like you can have somebody with the high school education and if they are rigorously forced to follow certain methodologies, I would probably be okay, but still believe it. Yeah. And or computer automated systems that are forced to follow those rigorous methodologies too, which is something that people are talking about. So it's been argued that the body of research behind forensic science is just totally incomplete. The methodologies aren't precise. Like I said, there's lying in fraud today. We're mainly going to concentrate though, on whether the science itself adds up so that you can take valid conclusions from it and use it as evidence in a courtroom. Right, pseudo science problems in the scientific method. Yeah, We've had chemists who have faked results and gone to prison for drug convictions. There's been sloppy work that's been done than thousands of cases related to Saint Paul, Minnesota, Detroit, Michigan, Philadelphia, North Carolina, Houston, and more. I mean, this is like, this is everywhere, but in two thousand five, this is this is the what led to the paper that we were talking about earlier. Congress commissioned the National Academy of Science to examine the state of forensics in the US in the U S law enforcement, and they this group found in two thousand and nine that there were quote serious deficiencies in the nation's forensics science system, and they advocated for extensive reform. And they said, apart from d NA, remember we're gonna come back to DNA later, there is no single forensic discipline that has been proven with a degree of certainty to be able to match a piece of evidence to a suspect. Now that brings up an important thing that I think may come up again. In this episode, which is that some fields, as practiced, may be better able to sometimes exclude the possibility of of a defendant being gilt rather than matching. If that makes any one example I know we'll talk about later. His bite marks. Yeah, and that that's something I've seen pointed out, is that while it might be hard to say, Okay, this bite mark identifies that this truly is the defendant, you could more easily say this makes it clear that this bite mark was not left by the defendants. Yeah. It's complicated, right, and it's uh, it brings me back, as many of our episodes do episodes do lately, to the wicked problems that Robert and I talked about earlier this year, that just what's going on with forensics in the justice system in general is a wicked problem. And I don't think it's gonna be solved with one answer, right, but but it's worth looking at. Well. I mean, one thing we can take comfort in is that this big two thousand nine report is very extensive, and I hope that this already has in some cases led to some initiated reforms, and it will help lead to continued reforms in the future. So it's not like nobody Reckon noises this problem and nobody's doing anything about. Yeah, I suspect that there are a lot of people who are working very hard right now. You may even be listening to the show who are like, hey, you know, we're doing our best over here. So we get that. But but it's also important to sort of walk through the process and figure it out. The worst implication of this, though, Joe is like, this is and and this is extrapolating it out to. One reason why a lot of people care about this now is because it's in our pop culture right Cereal, the biggest podcast out there, Making a Murderer on Netflix, all of all of which touch upon these things. There's an implication that if the forensic science is wrong, that there are innocent people who are jailed for crimes they didn't commit, and subsequently there are guilty criminals out there who are still roaming free to so, um what we're going to talk about this group a lot as well. The Innocence Project found that when prisoners were exonerated by d n A, the real perpetrators were identified in half of those cases, and in all but one, the real perpetray Waters continued to commit crimes, serious crimes after the innocent person went to jail. Yeah. So the Innocence Project, if you haven't heard of it, it's an advocacy organization, So it does you might say, probably have an ax to grind in this issue, but but it's also yeah, it's uh. I think it is abundantly clear that that many innocent people have been convicted of crimes they didn't commit. Many guilty people have been let go on the basis of bad forensic evidence, or or have been let go simply because somebody else has made to cop for the crime that they committed. Um, and yeah, and these people so so somebody who actually did a murder is out roaming the streets because somebody else is in prison for it. Yeah, And that's just one disappointing but too terrifying, right, uh. And so another part of the problem. And I don't think we're gonna have a lot of time to dive deep into this today. But it's also important to remember that the legal sciences don't get as much federal funding as other disciplines when it comes to research like this, so subsequently, there aren't as many examples of statistically defensible research that these forensics. Forensic examiners can rely upon that they can go back to and that they have a depth of research in their field. Right, And so one argument is we should fund them more and too they should be broken out into their own division. They shouldn't be beholden to police and prosecutors because right now most forensic scientists work under the police and under prosecutors um, which is obviously going to throw bias into their work. Yeah, that's another thing that is that is addressed by that big two thousand nine document is is experiment or bias? I mean, this is a problem that's also different in some sense from methodologies. Yeah. Absolutely, Well, actually, I guess you could say it's part of methodology because when you design an experiment in science, you part of your methodology is to try to remove bias, for example, by blinding or experiment or blinding people carrying out the research shouldn't be aware. Let's explain what that means. Yeah, you're not blinding a human being, right, experiment or blinding for example, is you know, if you want to test whether a certain type of artificial sweetener causes cancer. When you're conducting the experiment, ideally the person performing the experiment, shouldn't know what the hypothesis is, shouldn't know what the samples they're using are. They should be sort of unlabeled and identified later by like numbers that could be matched up. So so that the if the experiment or has a certain way that they maybe even unconsciously want things to go in the experiment, that can't affect it because they don't know what's going on and what's ex there's no actual variables that they could unconsciously put into the into the test, right. Yeah, but in many cases in forensic science, that's not necessarily how it's practiced. A lot of times the the forensic investigator in certain scenarios might know what the hypothesis is. It's that this certain person is guilty and here's what they did. But again that's one that varies from field to field, so we can't say that that's problem across the board. Yeah, So, like I'm thinking like an example here, although I'm not I don't want to accuse the woman in the case that I was a part of of of pseudoscience in any way. I have no idea one way or the other. But her job was to prove that the cocaine that was given to her had a certain percentage of pure cocaine in it, so that this man could be charged for distribution. There's like on the law, it has to be a particular percentage so that it's legally enforceable. Um, you know, rather than it being like, I don't know salt, I don't know what people mix in with their cocaine nowadays, sorry, guys, not on the street as much as I used to be. Yeah, non dairy creamer and salt. The fair powdered here. Yeah, Um, but she knew what her job was going into it when she was measuring right. So yeah, you mentioned that word pseudoscience, and I guess we should get into that a little bit. Uh. It's something that comes up on this show fair often we talk about pseudoscience, But what what is pseudoscience for? For those who are you may have heard the term, but you're not clear exactly what it is. You're you're aware that it's maybe just false information or something. But I think that in this case it's very important to emphasize the specific definition of it because pseudoscience US standard Dictionary definition I found is a collection of beliefs or practices mistakenly regarded as being based on the scientific method. So um, the scientific method with reference to forensic science, as defined explicitly in that big two thousand nine UH National Research Council report is quote hypothesis generation and testing. So you've got a hypothesis going in falsifiability and replications, so there are ways that you could show your knowledge is not true, and testing more than once by different people to verify the results you get. And then peer review of scientific publications, so testing your findings against other experts in your field to see what they think and if they can criticize your method. Now, other than the peer reviewed aspect, this is pretty much what we learned in science class in high school, right, well, depending on Yeah, when you get your little research lab book and you're filling out all your stuff for the experiments that you perform in class and such like that, this is what they're teaching us. The importance of that the peer review stuff comes later when you're in I guess higher education. But uh, it shouldn't be that hard, you're right, But in a lot of cases that this is exactly what they have found. Many forensic fields, or certain forensic investigators don't always practice that their techniques don't reflect this, or the rules of analysis they use don't necessarily reflect this. They're not based on well replicated, falsifiable scientific tests. They're more just kind of like knowledge, you general wisdom, folk wisdom about here is what you'd expect to see here, So kind of like, so I like to think about a scene that I love from TV. But this is total pseudoscience, all right. But it's also like street cops right the wire, the infamous f words seen in the wire, you know what I'm talking about, Right where bunking Meltier in the kitchen, and they're trying to figure out how a woman was shot through a window in the kitchen, and so they're basically measuring with pens and like I don't even know if they have a measuring tape in that scene or not, you know what I'm talking about. And they're measuring like what the trajectory of the bullet might have been, what angle the guy might have been shooting from, where she was shot, where the bullet might have been lodged in the refrigerator door, all this stuff. In the meantime, while they're swearing a bunch, which is funny for us as the audience. But that's not science. Well, I mean, that's just them. They've got some common sense and yeah, they've they've been on the job, they've seen enough homicides that they have a bit of an idea on how to explore that scene. But that's not something that unst necessarily would submit to the courtroom, right, I mean, I can I can look at that scene and say, I don't know, it seems like they're reasonable in the conclusions they're drawing. Yeah, but um, but yeah, I mean one problem there is, for example, the lack of quantitative evidence. Yeah, this is something that's often important in science, is trying to find a way to represent your findings with numbers so it takes the gut feeling out of it. So instead of saying, yeah, that bullet hole looks like it came from there. Instead what you should be doing is putting a straight object through it and measuring with a pro tractor exactly what the angle is, so that it takes your gut feeling out of it. You have a specific number that has been explicitly measured. Now, I think if David Simon, creator of the wire, we're here with us, he would argue that one of the points of that scene also is that because the Baltimore Police Department is so horrifically underfunded, this is how these police have to go about doing their jobs because they don't have protractors or forensics teams that can come in and do all that for them. Right. So, one important point I think that we should make is that pseudoscience isn't just any false knowledge or bad epistemology. It's a specific type of thing. Like if a prosecutor on a case, you're on a jury, and prosecutor brings in a tarot card reader, uh to turn some cards to show you show you exactly how guilty, Uh, you know this guy is of attacking somebody with a folding chair. That would be a bad evidential standard. But I don't think I wouldn't the tower came up, I'd be real worried. Huh. I wouldn't call it pseudoscience because it's not trying to dress itself in the wardrobe of science. And that that's exactly what is so pernicious about some of these flawed forensic science methods. They take on the credibility of the scientific method without actually practicing the method in every case. Yeah, and depending on how charismatic or rhetorically qualified uh, defense or prosecutor could be. You know, it's very easy to convince a jury that somebody is an ex it in something that they just simply don't understand, you know, so that's also worth considering. Yeah, um, all right, I want to bring up one thing here that references another old episode of ours too, which is um, Robert and I did that episode earlier this year about cargo cult science and our cargo cultism, and then we also talked about cargo cult science with a concept from Feineman. H yeah he was he semi involved in that. Yeah. Yeah. In fact, we told a really fun story. Go back and listen the episode. Everybody but told a fun story about Fineman hanging out in a bathtub while people were doing reflexology on each other. Uh anyway, Fineman always hanging out in bathtub? Yeah, exactly, hot tub, I guess more than bathtub there. Um. But so this also it made me think of what Robert and I talked about as the Church of science quote unquote in that episode two, that there is there is a an idea of science as being almost like a deity now days, right, that that that true knowledge and true um ways to judge how the world works can come from science rather than the old way that we thought it did, which you know generally was from religion or mythology. Uh. And this is an example of where quote unquote science is a false god. Uh. You mean, like the people who treat science sort of as dogmas rather than as a method. Exactly when people say, I hate when I hear this, when people say quote science says, yeah, that's one of my least favorite phrases. Science doesn't say anything. Science is a tool. It's not a it's not a deity speaking pronouncements to you. And one of the people one of the things scientifically literate people will know is that, um, you know, scientific results are tentative. You you learn something through science, but then there might be new studies coming out next year that say, actually, those previous studies were flawed and here's the better answer. Yeah, I guess. Just my general point is that, like, it's important to realize that this is fallible and that you should apply critical thinking to this when it's when it's put in front of you as matter of fact, right, knowledge that is cloaked in the garments of science ain't necessarily so just because it's wearing those clothes. Absolutely, So Okay, I want to bring this back though, because I don't want to make it seem like we're just totally bashing forensic science and it's all bad. No were to put your critical lenses on, and there are examples of well researched science being used in forensics. In fact, chromatography which was one of the things we were going to do for the We're gonna do paper chromatography for our little d I y uh lab experiment. It's a method for separating complex mixtures and it allows examiners to identify bodily fluids for drug cases, and for the most part it's seen as being well researched and backed up. Uh. The other one which we mentioned is DNA analysis, which has set a new scientific standard. But if you also have to remember that quote, good science takes time, right, so you can't just like whip up DNA evidence like you know, in a flat Literally the TV show The Flash, I don't know if you know this, the character the Flash is a forensic scientist, and so whenever he's like, you know, got work to do, he just super quickly like does it all like in like thirty seconds or whatever. So it's that's not a good example. Why does he make the machines work faster? No, he just runs around really quickly and like shakes like like like little pipettes and stuff or test tubes really quickly. The machines work faster because he is the machine. So he becomes a centerfuge or something like that. Yeah. As much as I love the Flash, both TV show and just as a character, it's like the worst example of what should happen in forensics. SciTE um. Anyways, the DNA thing, it took thirty years from the discovery of the double helix structure before it could actually be used to confirm a positive identification of an individual. Now it's broadly accepted and quantifiable, right, but it took a long time. In fact, the most advanced analysis has a one in more than a quadrillion chance of a random match of two strangers DNA, So that seems fairly reliable. Yeah. Yeah, Yet DNA only constitutes less than ten percent of the case load in US crime labs, right, So that's important to remember as well. What we really need is the rest of forensics to be just as rigorous and justice statistically grounded. Yes. Uh. And one thing I do want to say, picking up on what you said a minute ago about how we're not trying to trash all these fields, is that I would say that I think any field can be pursued in a scientific way. So if we say something about major flaws in uh, bite matching or something, or in fire analysis, that's not to say that forensic ode ontology, this study of you know, dentistry in in crime cases, or that fire analysis are not legitimate fields of study. There are totally legitimate ways to study these, and there are lots of great scientists who do exactly that. The problem is that as practiced in many in many court cases and forensic investigations, it's full of bad methodologies or unverified knowledge. Yeah, and I think you know that c s I effect that we were talking about earlier too, is perpetuated by by way more than just like the sort of um police procedurals like c s I that we're used to. I mean, like I just mentioned the Flash that's not really a cop show, and yet like it throws in a dose of that kind of stuff in there. Also thinking of like a great TV show, Sherlock but it relies on like lots of like examples of Sherlock Holmes conducting his own like at home forensic tests. Have you ever seen this show before? Like I've seen was one where like he's just like beating a corpse with like a horsewhip or something like that, just just to see, like what how long it takes bruises to form on a corpse? Right, And it's like, Okay, like I get it that that's an interesting like way narratively to show us that he's conducting research and everything. But also like what but it's funny that Sherlock is often used as a as a you know, great symbol of scientific investigation because he uses deeply unscientific methods a lot of the time. Sherlock Holmes will you know, look at a few facts about you and make deductions, right, That's what he does science of deductions. Say well, your watches turned back to this time, which makes me know that you were in this country and this time zone. That's not scientific, that's just making an assumption. But we love it. We eat it up right, like as an audience. That's great fun because it makes the world a lot simpler than it actually is. And it man like, of course, like I would love it if, like I could be Sherlock Holmes and just solve every problem by just like kind of very quickly looking around the room and knowing everything that's going on. But it's not realistic. It's a fun story, but in terms of like people's lives or whether or not they're guilty perpetrators out there running around, it's not really how things work. Okay, we need to take a quick break, but when we come back, we're going to talk about evidence standards for scientific evidence in US courts and then a bunch of specific examples of forensic science disciplines and what some problems with them might be, including, uh, the aforementioned case of phrenology. Okay, we're back, So Joe tell me and the audience what is the fry test and how how why is it pertinent to this forensic research that we're talking about here. Well, this comes up in a lot of discussions of forensic science because this has been the legal standard for the admissibility of scientific evidence in US courts for a long time. Now. There are other standards that have in some ways superseded it, but this was sort of the original one. So The Fry standard essentially determines that for scientific evidence to be admissible in the court, it has to be quote generally accepted by experts in the field in which it belongs. So Fry came out of a murder trial in nineteen three and which the defendant tried to demonstrate his innocence by using a lie detector test to measure systolic blood pressure. So this is an early lie detector test. And and this guy is going to say, look, you know, I didn't do it. And here's a machine that shows I'm telling the truth when I say that I didn't do it. Yeah, you know what. I think. This is before the polygraph, because William Marston invented that, and he is also the co creator of Wonder Woman, the fun things that Christian Sager knows in his little weird brain, the lasso of truth, Yeah exactly, And that didn't happen until the late thirties early forties. That's a whole other field of forensic pseudoscience that that we can talk about. But so in this case, the court rejected this evidence, writing quote, just the scientific principle or discovery crosses the line between the experimental and demonstrable stages is difficult to define. Somewhere in this twilight zone, the evidential force of the principle must be recognized. And while courts will go a long way in admitting expert testimony deduced from a well recognized scientific principle or discovery, the thing from which the deduction is made must be sufficiently established to have gained general acceptance in the particular field in which it belongs. And that's where this general acceptance doctrine comes from. It So, lie detector tests not generally accepted by the relevant scientific fields then or now, really, and yet we still see them in pop culture all the time, right, And even if they were mostly accurate, they're not legitimate scientific evidence to use in a court. But even this is not fool proof depending on how it's applied and interpreted. For example, what if a whole field is just rotten to the core with pseudoscience. Um. So, again, it might not come as a surprise, but most phrenologists consider phrenology a legitimate science. Yeah, of course they would. And so if you don't, here we go, Yeah, here we go. I see the next line coming up. I'm gonna I'm waiting for the emails to come in. Right, most people who own orgon energy accumulators probably think Wilhelm Reich is not pseudo science. Yeah, absolutely so those of you who are long time listeners may know. Earlier in the year, Joe and I did an episode on Willhelm Reich and organ accumulation and cloud busting, and we received some nasty emails from people who are deep followers of Reich's belief. Yeah, they were not happy that we did not see the genius in his method. Yeah, and I I like to feel like we did a fair treatment of that topic, but that they were upset. Yeah, well, you know, we do our best, but but I don't know. As we see, I don't think the organ accumulator has much going on for it. Yeah, and neither do I. And I'm the one who owns VHS tapes and books by that guy and has state at his estate. Uh So, in these cases Wilhelm Reich phrenology, it's obvious to us what's wrong, because we're well aware of the these fields and their faults. But what about in obscure sciences, you know, things that are not popularly known to be uh, pseudoscience, where an entire field could just be riddled with problems and nobody on the outside would know UM. So in nineteen the Federal Rules of Evidence were established, and this brought in Rule seven oh two which is relevant to this, and it reads quote, if scientific, technical, or other specialized knowledge will assist the trier of fact to understand the evidence or to determine a fact in issue, a witness qualified as an expert by knowledge, skill, experience, training, or education may testify there too, in the form of an opinion or otherwise. So this seems to massively just on the face of it, this looks like it massively relaxes the standard um and and this led to much argument, lots of back and forth between legal scholars like does this embrace the Fry standard or does it reject and replace it? And there were many fierce arguments, but eventually Fry was superseded by the Daubert standard in the courts. So the Daubert standard is much more complex and it says um Essentially, the qualifications for the admissibility of scientific evidence in the courts are whether you can test the theory. That's an important thing, like it shouldn't be just unfalsifiable knowledge that there's no way you could show if it was wrong. Second, whether it has been subjected to peer review in publication. That's another important one. What it's known potential error rates are. So you know, you could have a totally valid field that has a known pretty high error rate. I mean, it could still be valid as long as you acknowledge what the known error rate is. Yeah, and acknowledge that to the jury exactly. Um, the the existence and maintenance of standards controlling its operation, right, so you've got to have uh systems in place to to make sure everybody's doing it right. And uh, and then again whether it is generally accepted among the scientific community. So who where did this come from? Yeah? Daubert was established in nine case in which the plaintiffs were attempting to show that a drug called ben Deckton had caused birth defects in their children and uh, and it clarified this rift between Fry and rule seven O two, saying that seven O two, as enforced, should have all those qualities I just mentioned, um, but also that that two thousand nine National Research Council paper that's like that's like the big daddy source that we keep coming back to. If you want to know everything about this. You should go read that very big. It's learning, it's huge, um, and that they write that quote. The court also emphasized that in considering the admissibility of evidence, trial judges should focus solely on experts, principles, and methodology and not on the conclusions they generate. And that seems that seems absolutely crucial and correct to me. What should be important is the method they use, not what they end up saying. Right, yeah, and that's still I would say, or at least just from my experience, not exactly the case, you know, but the two thousand nine documents says that. I mean, it's been seven years since then, so hopefully it's better. But I think that case that I worked on was probably a couple of years after this. Yeah. But another thing that this, uh, this indicates is that the court essentially placed faith in the system by saying, look, we've got an adversarial system. So you have prosecution and you have defense. You have plaintiff and you have defense. Since there are two sides, we can be sort of generous in what kind of scientific evidence we accept. Because if the defense doesn't like the prosecut uter's expert witness, they can call in their own expert witness. Yeah, and so the court wrote, quote, vigorous cross examination, presentation of contrary evidence, and careful instruction on the burden of proof are the traditional and appropriate means of attacking shaky but admissible evidence. That essentially means it's not the court, or rather, it's not the judge's responsibility. Uh, it's this. It's on the responsibility of the prosecution and the defense to argue rhetorically against the science. Yeah. So the judge might have the ability to rule out a phrenologist giving evidence, but in a case where that's not as clear as phrenology, but just where the science appears legitimate but might be shaky. Uh, it's incumbent upon the lawyers, right that We'll let it in and we'll see if the other side has something to say about that. All right, So, phrenology, you've been you've been teasing around for knology. I know you've been wanting to tell this story all episode. Okay, So I'm gonna take you back more than a fifty years. I'm gonna take you the thirty four the United States, in the state of Maine. We're up where Wilhelm right, Well, he wasn't doing an eighteen thirty four but yet right unconnected early rangely Maine. Yes, this is will not ring unconnected to Wilhelm Wright. In Maine in eighteen thirty four, a nine year old boy named Major Mitchell was facing trial for a serious violent offense. He was charged with assaulting and maiming another boy, an eight year old schoolmate named David Crawford, and according to the allegations, Major Mitchell had lured Crawford into an empty field with the intention of whipping him and killing him. H Crawford had called Mitchell names like a hog, a fool, and a steeler, and then Mitchell began to punch Crawford until an adult neighbor came by and separated them. Later, Mitchell found Crawford walking along a road and forced him into some nearby woods. There, Mitchell began torturing Crawford, he uh supposedly a acording to the allegations, He filled his mouth with grass, stripped his clothes off, tied him up, beat him with sticks, tried to drown him in a stream, including damning the stream to make the water deeper to make it easier to drown him, and partially castrated him with a piece of rusty taka. And this is what happens as you name your child major. This is messed up. This is not yeah, not a happy circumstance. That's a joke. By the way, there's no evidence that naming your child major will scientifically lead to them being a scumbag. But eventually, but this is a nine year old kid. This is something weird going on here. This would be extremely disturbing if an adult did this to another adult, but this is a kid. Doing this to another kid is just so messed up and um So, eventually Mitchell let Crawford go and Mitchell's legal defense was taken up voluntarily by this Portland's lawyer named John Neil. Now Neil didn't just do this out of like pity or kindness. Neil happened to be a proponent of the budding science quote science and quotes there of phrenology. So for for the audience out there, and this is my imagination of phrenology at the time. I have an inkwell at home that is a bust of a head and it has all the phrenology. You know that the stereotypical like phrenology drawings of like which part of the head does, which it contains which emotions. So those of you out there who are wondering what does this phrenology they keep talking about, think of those like those illustrations of like a profile of a head and it's like carved up kind of like kind of like those like depictions of like what parts of a pig are good to eat? Right? Yeah? Yeah, and it shows like okay, and this part this is where anger is and in this part, this is where compassion is. Oh yeah, it's great. So, uh, it's not great, it's awful, but it's great. It's great fun to look at it. So phrenology might in a primitive way be considered a predecessor to neuroscience, and that it linked mental faculties and personality traits to the physical makeup of the brain. Uh, who you are as a product of your brain. That's not a bad starting place for for science. But from there it sort of becomes the body builder bro science of the brain. Uh. So you know how when you lift heavy weights use certain muscles, those muscles get bigger over time. I have no idea what you're talking about, at least in theory, at least in theory. Uh, you know, and so you can often tell how strong a certain part of a person's body is by looking at how big the muscles there are. Well, phrenology sort of takes the same principle to the brain. It positive that the brain is covered in these topographical regions. Quote organs, organs right, that are responsible for particular aspects of personality your behavior. So if you had a lump or a raised contour on the part of your head that chronologists legal labeled the organ of hope or the organ of secretiveness, those personality traits would be super pronounced in you. So if one of these guys you know, pulled the let me tell you to the gun show line, the guns would probably refer to something like the organ of benevolence, let me show you my head bumps. So like what what I mean phrenologists. Even at that time, people knew like if you get hit in the head with a baseball bat, bump forms on your head like oh like, depending on where you got hit in the head, your momentarily turned into a more benevolent person or a more aggressive person. That is almost exactly what the defense argued in this case. Okay, hit it so phrenology is now considered completely discredited. There's no evidence at all that these cranial organs had any validity. Uh, And you can't accurately predict people's behavior by measuring bumps on their heads. But the defense in this case argued that Mitchell had suffered an injury to the skull when he was young, quote, whereby the portion of the brain called by phrenologists the organ of destructiveness was prematurely enlarged and a destructive disposition excited. Man, if it was that easy to make children into little monsters, right, like you'd be able to very easily like create like an army of homicidal maniacs, but just like whacking them on a certain part of their head. Well that that's not actually completely off. And I want to get into that in a minute. Because so in this case, they observed a lump behind Mitchell's right here, and they said, look, you know, the organ of destructiveness is enlarged. Uh. The crime was a result of this enlarged organ of destructiveness. The enlargement was due to circumstances beyond Mitchell's control. He just got hurt. Um, so he shouldn't be held responsible for the crime. And the judge wrote, quote, where people do not speak from knowledge, we cannot suffer a mere theory to go as evidence to a jury. So the judge actually pretty wisely in this case ruled like this, This like old main judge, though it sounds like people do not speak from knowledge. It's like in pet cemetery. Yeah, exactly, you don't want to go on that road of phrenology. That's a bad road of phrenology. Okay, yeah, those are a terrible main accent. I'm sorry. No, mine is not just as equally bad, and I'm from New England. But this does raise an interesting question like how to outsiders and lay people determine what is knowledge and what is mere theory, and how to judges do it by the preferred nomenclature of today, I think most scientists would prefer to say mere hypothesis than mere theory. But either way, um, neither judges determining admissibility nor jury members listening to an expert witness testify typically have exhaustive knowledge of these disciplines. So what do you do? Yeah? But then again, I do think this case raises important issues that are still relevant today because, for one, for one thing, courts today will take real evidence about, say injuries to the brain or illnesses that affected the brain into consideration when thinking about somebody's responsibility for a crime. What if you, you know, normal person, not a violent person, suddenly one day get the urge to go out and start beating people down with a folding chair. And then they discover that you've got a tumor on your brain rick flair disease. The the this tumor may well be changing your behavior in a way that it's kind of hard to hold you personally responsible for, right, I mean, that's that's shaky, but yeah, well, I mean that's how a lot of the courts would look at it. And I'm sure you'd feel the same way if you were, Like, you'd be like, this isn't me. I don't know why I did that. That doesn't feel like a part of me at all. And then they discover, well, there's a tumor in your brain that's changing the way your brain works. Can you remove the tumor? Well, maybe you can, maybe you can't. Yeah, I mean I guess like whether or not you're responsible, you're still responsible. But then then it changes, like what the punishment would be right, like, like, you wouldn't necessarily send somebody like that to jail, but they would need to be ice lated so that they're no longer doing this if yeah, I mean hopefully they could get treatment and other things would be a brain injury. You've probably heard the story of Phineas. I was just about to bring up Phineas. We talked about him on the show all the time. Phineas Gage. He's this guy. What what you're with that? Do you remember? I can't remember, but I do remember that when I was in a junior high school that our school had a DARE program and they would often talk to us about the repercussions of drinking and driving, and they gave us all t shirts that had a picture of a skull with a metal rod shoved up through it, and it was to remind us about Phineas Gauge and how his brain changed because of the rod. If you're not familiar with this incident, he was a railway worker working on a railroad and and there was some accident and explosion shot a metal rod through his head and his personality completely. Yea, he lived, It didn't immediately kill him, but he was not the same man. And this brain injury had altered something physical about the way his brain works, and he was no longer morally or in terms of character, the same person. Right, And so like in my instance with these T shirts, Dare was trying to use this as a metaphor for like, this is what happens when you drink is you're no longer the same person. Subsequently you shouldn't drive, right, Okay, But but this is another way that I think neuroscience may come into forensic science in the future, because like, the more we understand about neuroscience, the more we understand about what things of physical changes in the brain can lead to certain behaviors with with pretty reliable predictive power, you know, you say, like, oh, we've discovered that most of the time you see this physical process in the brain, people start behaving in this way. Uh. Certainly, especially like when you're talking about depression and anxiety. Yeah, exactly, do those things start changing our idea of how responsibility works in the courtroom? Uh? So, Like if we say that you know, somebody who Phineas Gage had a bar goes through his head, or somebody had a tumor in their brain, and this directly seemed to lead to a change in their behavior that caused criminal activity or something like that. Uh, and what what if it's not an injury or an illness but just some inborn condition. This is just how your brain was born. Yeah. Yeah, it's complicated stuff again, wicked problems right, Like like it's not it's not even just as easy as like having the forensic science be perfect, because even then you get cases like what you're these hypotheticals that you're putting out. Yeah, it's very complicated to decide what's right and wrong. Yeah, and since we ran so long, we're gonna have to call it right there. So here, here's what we're gonna have to leave off the first half of this discussion, but to hear the rest, you can simply tune in again next time. And in the meantime, you can find us on social media such as Twitter, Facebook, Instagram, and Tumbler, where our handle is some variation on blow the Mind. You know what kind of weirdos we are. You'll be able to tell if it's our accountant, I'm quite sure. And of course you can always catch always say and do. It's stuff to blow your Mind dot com. And if you'd like to email us to get in touch with ideas for future episodes or feedback on this one. You can reach us and blow the mind at how stuff works dot com for more on this and thousands of other topics. Is that how stuff works dot com. Remember remember,