In this classic episode of Stuff to Blow Your Mind, Robert and Joe discuss regression to the mean, a phenomenon that occurs when a random sampling point is an outlier -- with applications to everything from economics and sports to health, religion and disappointing sequels. (originally published 8/3/2021)
Hey, welcome to Stuff to Blow Your Mind. My name is Robert Lamb and I'm Joe McCormick. And it's Saturday. Time to go into the vault. This episode originally published on August three, and it's called Regression to the Mean, which is about regression to the mean. Uh, it's an episode about a statistical phenomenon. But don't don't don't run away. It's actually, I think super interesting and having this idea in your toolkit really helps you understand the information that you encounter in the world, uh much better. Absolutely, So let's dive right in. Welcome to stot to Blow Your Mind production of My Heart Radio. Hey, welcome to Stuff to Blow Your Mind. My name is Robert Lamb and I'm Joe McCormick. And in today this episode, we are going to be focusing on a topic that is already something that's very well known to people who are familiar with quantitative research and statistics, but less known to the general public. And uh, and I think that's a tragedy because it's an idea that should really be part of everybody's basic critical thinking tool kit, no matter what your job is. And so in order to introduce this concept. I thought it would be best to start with a with a direct illustration from the real world of people reaching incorrect conclusions by not understanding the subject of today's episode. And so the illustration I want to start with is an interesting story told by the psychologist Daniel Kaneman that's about the illusory power of screaming at pilots. Uh So, the context of the story is that Knemon says he was giving a lecture about positive reinforcement to a group of flight instructures. I think this was in the nineteen sixties, and Kneman was trying to inform them about what he believed at the time was the best consensus of scientific research on learning and reinforcement, which was at the time that if these flight instructors wanted their students to have the best possible outcomes, they should focus more on praising the students when they did well, then on chewing them out when they did something wrong. And Knomon says that when he finished his talk, one of the flight instructors that he had been giving this lecture two got up and tried to dispute him. He said, no, you're wrong. And so the direct quote economy and gives from the instructor here is, on many occasions I have praised flight cadets for clean execution of some aerobatic maneuver, and in general, when they try it again, they do worse. On the other hand, I've often screamed at cadets for bad execution, and in general they do better the next time. So please don't tell us that reinforcement works and punishment does not, because the opposite is the case. So you might think he has a good point here. If you accept that this flight instructor has had a lot of direct experience working with students, and you trust him to remember the relative frequency of these events pretty well, you might assume that he has a meaningful rebuke to konom In. Here again, he says that most of the time, after a cadet does something bad and he screams at them, they do better the next time, And after a cadet does something good and he praises them, they actually do worse the next time. So if he's remembering these experiences correctly, and he's had a lot of them, it would really seem like evidence that praise has a negative effect on learning, maybe by making the student pilots soft and overconfident. Or something, and getting chewed out is good for skill development. I think it's quite easy to see the allure of this, this false conclusion right right, And it's and you can also easily imagine how you kind of build upon this with certain loosely backed up you know, folk ideas about how you encourage people and how people learn, and you got to stay on and if they if you tell them they're doing a good job, they'll get lazy, right, folk wisdom, tough guy mentality. But Koneman saw something different in this response, and he says that he immediately set up an experiment on the spot to demonstrate the flaw in the flight instructors thinking here, so I want to read from Knomen's description, He says, I immediately arranged a demonstration in which each participant tossed two coins at a target behind his back without any feedback. We measured the distances from the target and could see that those who had done best the first time had mostly deteriorated on their second try, and vice versa. But I knew that this demonstration would not undo the effects of lifelong exposure to a perverse contingency. So to explain this, this experiment a little bit better. Right. He has people stand with their backs to a target so they couldn't see it, and they would take two a attempts to throw a coin and hit the target without any feedback of any kind. So they're not getting praised, they're not getting chewed out, nothing. Uh. And after staging a number of these, he found again what he suspected, that the people who were the closest on the first throw did worse on their second throw, and the people who were farthest away on their first throw tended to do better on the second throw. So what condiment is actually demonstrating here is something that doesn't really have anything to do with learning or reinforcement, or really skills or even human psychology. Instead, this demonstration is showing the effects of chance, luck, and statistics. What he was showing is the subject we're talking about today, regression to the mean. Uh. You'll you'll see that phrase a lot in in scientific literature and in statistics. But if it helps to put it in more everyday terms, anytime you see regression to the mean, you can translate it in your head as trending toward the average, trending toward the average. So to make the coin tossing illustration even clearer. Imagine you throw the coin not twice, but that you throw the coin a hundred times. So you stand there throwing the coin a hundred times. And then let's say afterwards you average together the distance from the target across all a hundred throws, and you'll come up with some kind of average distance from target. Uh, just to make up a number for the sake of argument. Common doesn't give this. But let's say the average distance from the target across all your throws is nine centimeters. And remember that you're getting no feedback at all here, so it's unlikely that you will be getting much better as you go on. So, given that the average distance from the target is nine cimeters, if you throw a coin once and it happens to be two centimeters from the target so really close, is your next throw likely to be about the same as that one, better or worse. Obviously, it is overwhelmingly likely that your next throw will be worse, just due to chance, probably closer to the average of nine cimeters away. And the same goes for throws that are really far off. You throw something three hundred centimeters off your next random toss just by chance is likely to be much better, much closer. So simply put, most of the time, if you're sampling something in a series over time, if one sample produces an extreme value, the next one in the series is more likely to be closer to the average instead of extreme in the same way. In my experience. Uh, this is this is why it can sometimes be liberating to start off a game of bowling with just a disastrous gutter ball, because because I know that I'm good enough that that's probably not going to happen twice in a row, but it's definitely going to happen at some point in the game because I'm not that good, you know. I put like playing you know, once a year or even with less frequency these days. Oh yeah. And also like why I think a lot of us have intuitions that when you try something for the first time and you do really good on the first attempt, that makes you kind of nervous because you just know you're probably not going to live up to that repeatedly. Yeah, like if you get if you get a strike that first time, then that that first um what is it round? I can't even remember. This is how one frequently I bowl, Um, the first role. So the first role first, the first column. You know, So, the tendency of regression to the mean or or trending towards the average is pretty obvious when you're dealing with something like lots of random coin tosses with no feedback, But it becomes much more obscure when you're dealing with, say, a more more limited numbers of outcomes. In the series, you're looking at and introducing possibly influential variables like pilot skill and instructor feedback. After all, we would expect that some variables having to do with instructor feedback should have an effect on pilots ill, right, That's the point of teaching is to have an effect over time, and after all, in this one scenario, the conomen describes the the instructor believed that his verbal abuse of the students was so motivating that it made them instantly better on the stick. And you can't necessarily rule that out, but it's unlikely. I think I'm convinced that regression to the mean could more easily explain this flight instructor's belief that screaming at pilots for screw ups made them better at planes, because, again, on average, even in the absence of any feedback at all. If a pilot in training executes a maneuver perfectly, the random fluctuation from one execution to the next will tend to mean that their next attempt probably won't be as good as that really good when the last time. And likewise, if they make a major error totally botch a maneuver, they're more likely to do better the next time just by chance. Both of these tendencies are regression towards the mean. But then Conomon actually draw is a really interesting observation about about about our psychology and about culture from this fact, so to quote him directly, this was a joyous moment in which I understood an important truth about the world. Because we tend to reward others when they do well and punish them when they do badly, and because there is regression to the mean, it is part of the human condition that we are statistically punished for rewarding others and rewarded for punishing them. And that was one of those things that when I read it, I was just like, oh my god, that's so true. Um, yeah, yeah, And in this specific instance, it makes me think about the special effect of reversion to the mean, fallacies on motivating belief in the effectiveness of of not just screaming at pilots in this one case, but all kinds of punishment behaviors, for example, corporal punishment. Thankfully you hear this less often these days, but I remember when I was younger, I used to hear people who would defice end the parental practice of spanking children by saying, you know, I don't I don't care what the site scientists say. I don't care what the research says. I know from experience that it works. To the extent that comments like this were based on any real experience and observation and not just sort of a free form, self justifying statement that had nothing to do with experience. I bet a lot of it was fallacious inference of causation actually based on regression to the mean, just like in this condiment example. But anyway, I thought it would be interesting to talk a bit more about regression to the mean today, because it's one of those things that, again, once you see it, it's it's pretty simple, it's actually actually pretty clear, but understanding it can help you have a better sense of how good science works and help keep you from drawing hasty inferences in everyday life. Yeah, because it is it is interesting how kind of an insidious the results can be the idea that that again, praise is ultimately punished because is there's going to be a regression to the mean, to to to to the mean, and then likewise there can be this illusion, uh that uh that's screaming at pilots and so forth is going to be the successful way to go about things. Um. So yeah, this is I think this is an important episode to cover because it's the kind of thing that it's the kind of tool you kind of need tucked in your back pocket, even if you're just doing something like like scanning science headlines on a you know, a news server or social media message board. Yeah, because of course, understanding regression to the mean is extremely important in what scientists do when they design good experiments. If you don't take into account regression to the mean, you can incorrectly believe you have discovered some kind of tiger repellent or something. Uh. This concern plays a huge role in the history of medicine. It's part of the design of good medical research, or really any field that seeks to find remedies for problems. So consider a very basic hypothetical, uh path medicines, say from a hundred years ago. So you know, you have you have a foot pain that you've never really had before. Uh, you know, you want it to go away. So you go to the store and you buy a bottle of doctor Field Grades No Fail Pantasy for tumors, ulcers, cramps, and rooms, and you you pull the cork out, you chug it, and then the next day your foot feels better. Now you can conclude from this that the doctor Field Grades cured you. But how do you know actually that the feelings in your foot didn't just regress to the mean because the average is a low amount or no amount of foot pain. And if you don't have a medication that's tested with control groups and and randomized allocation into the groups, then how do you know that that the medicine actually did anything at all? Yeah? Yeah, So many of the examples you see for this and the applications, you're dealing with some sort of situation in the world, whether there is fluctuation and or change happening, often separately from whatever is being tested. So in this case, yeah, the doctor Field grads could have just been like just water. It just just you know, but there is the illusion that it worked because things got better. But if you don't have a control group and to you know, to drive home what that is. That would be like if you had a had like three different groups and a study of doctor Field Greats elixir. Here, one group was taking doctor Field graades elixer, another group was taking I don't know, let's say a half dose of Feel Grade or maybe a competitor's tonic. And in one group the control group was taking nothing was or was taking you know, just water or something to that effect, something completely innate. Uh. And that would be that would be a group that you would judge the results of the other categories by, right, and you would need to randomly sort the people into those groups. So it wasn't just that, you know, the only the people with real severe foot pain we're taking the doctor Field grades because the more extreme their pain to begin with, probably the more likely they are are to have that pain be lessened or go away over time, just naturally. Right. And uh. And I'm going to have a more specific example of this a little later in the podcast. So if you if you still don't get it, just hang on we'll we'll have another example in a bit thank. I was looking at an article in the British Medical Journal from nineteen that was just a collection of different examples of regression to the mean in real life medical research. This was by J. Martin Bland and Douglas J. Altman called statistics notes some examples of regression towards the mean, and they point out a very common type of example. So this will be similar to what we just talked about. The author's right. In clinical practice, there are many measurements such as weight, serum, cholesterol concentration, or blood pressure, for which particularly high or low values are signs of underlying disease or risk factors for disease. People with extreme values of the measurements such as high blood pressure may be treated to bring their values closer to the mean. If they are measured again, we will observe that the mean of the extreme group is now closer to the mean of the whole population. That is, it is reduced. This should not be interpreted as showing the effect of the treatment. Even if subjects are not treated, the mean blood pressure will go down owing to regression towards the means. So again something starts with an extreme value in certain types of cases, you would just expect it to have a less extreme value the next time due to random fluctuation. Uh So again, you know this could fill you with despair because you might wonder, well, then how could you ever know if a treatment was effective or not. But again, this is where the standard practices of science based medicine come to play. Instead of just taking people with some extreme measurement and giving them a treatment, you randomize them into test groups and control groups, like we were just talking about. So if you have a large enough sample, you really randomize the groups. People with the extreme starting conditions will somewhat regress towards the mean, but they will all regress toward the mean on average the same rate, whether they're receiving a real potential treatment or they're in the placebo group. But if the treatment actually does something helpful, this effect will manifest as the difference between the two groups. So good scientific research, good medical research has methods for excluding the effects of reversion to the mean on their findings. We have the tools, but we can still fall into the trap of regression to the mean fallacies, especially in our day to day lives. Drawing inferences the way that that the pilot and Inconomens story did, or or even in science if we're not careful and deliberate about designing experiments. And in addition to just a methodology design that has you know, a randomized groups and control groups, there are also ways of trying to counteract regression to the mean, just through statistical methods that are maybe less reliable, but there are statistical methods people can used to try to apply sort of modifiers to data in order to estimate regression to the mean and uh and counteract its effects. So again, we have tools within scientific research to to figure this out, and it's a lot of what science does is trying to sort out the difference between regression to the mean and actual effects of interventions. But in our day to day lives, we still fall for regression to the mean fallacies all the time. Yeah, and it's important to realize too that it's not just a situation where regression towards the mean could create an illusion of something working when it doesn't. Uh. You know, sometimes it can just potentially overstate um the effects of something. For an example of that that I was looking at was that regression towards the mean, or the failure to account for it can also overstate the effectiveness of something like traffic light cameras. Is it making a difference and cutting down on accidents? Perhaps, but any actual effectiveness could potentially be overstated by failure to account for just regression towards the mean. Oh yeah, so where do you tend to install things like that? High acts like problem areas? Right, So, if there's like a stretch of road that has a lot of problems on people really speeding a lot there or crashing a lot there, that might be where you stage the intervention. It's possible some things like that fluctuate naturally over time in different locations, and you put the cameras in place, and it could have an effect, but maybe not as much of an effect as it looks like it is taking place. Again, if you don't factor regression towards the mean into the study. Right now, While our TM is a very important phenomenon to understand and take into account, it certainly doesn't apply to every sequence of values you could repeatedly sample, so you also have to be careful not to apply it in situations where it isn't warranted. I was you know, there are a million examples. You could cite one that came to my mind as the orbital decay of a satellite. Let's say you've got a communication satellite in lower orbit and you get a reading on its altitude and the reading is lower than the satellites average altitude. Uh. Now you might say, hey, I think this means we need to program a reboost to insert it back into the orbit where it's supposed to be. And somebody could erroneously apply regression to the mean here and say, nah, we don't need to do that. The satellite might just return to its average altitude. It doesn't apply in this scenario, even though you are taking repeated measurements of a value over time, because we know things about the physical characteristics determining the orbit of satellites and in lower th orbit uh, and that due to factors like atmospheric drag, their altitude tends to trend steadily downward over time in a consistent direction down down, down, So eventually you will need a reboost in order to put it back up to the correct distance. So regression to the mean apply is to certain kinds of data that are repeatedly sampled data where there is natural random fluctuation back and forth, not a steady trend in the data in one direction on the relevant time scale. The other thing that's important to understand is that systems where you expect to find regression to the mean are systems in which the repeated data values you're sampling are to some degree determined by luck or chance. If a series of values is influenced almost entirely by deterministic influence, like in the satellite example, by like the laws of physics, or by some extremely reliable skill with little room for variation, values don't really regress towards the mean in the same way because there's just less random fluctuation back and forth to begin with. The more chance and random variation plays a role in the outcome, the more you will tend to observe regression towards the mean after an extreme sample in in whatever it is you're looking at, I've I've read that the progression towards the mean is is not to be confused with the law of large numbers. For example, uh. This is the the law that that states, as a sample size becomes larger, the sample mean gets closer to the expected value. So a coin flipping example is key here. Flip a coin and the random results are going to ultimately average out to a point five proportion. But if you only flip the coin ten times, you might not see this breakdown. Um. And this also applies to say, even odds on the rolling of a of a D six of a six sided die. Uh So for example, two regular people, that's just to die that nerves like us, it's a D six. Yeah. D six is what I could get my hands on. Because I was like, well, I'm gonna do an example. I'm gonna try it myself. So while I was putting together notes for this, I went ahead and rolled ten times, and I got even even odd even odd even even even even odd. So that's that's seven to three in favor of even. So it might make you wonder, well, is this die broken? Does this D six need to go away? Because it can't be trusted to roll? Uh? You know a balanced array of odd and even numbers. Well, no, that's not the case. Uh. And if I were to roll this, say another hundred times, another thousand times, I would see things even out even more to where we would see this, uh, this point five proportion of odd versus even right. So these are not exactly the same thing, regression to the mean and the law of large numbers, but they are closely related. Both observations require you to think about statistical tendencies over time, over a time period of repeated sampling, and both are premised on the knowledge that repeated samples will tend towards the average. But regression to the mean has to do with the idea that if you start with an extreme observation and there is some role of chance or luck in determining the value of this observation, the next time you sample it, it's more likely to be closer to the average. The law of large numbers is that if in the real world, the more times you run something, the closer your outcomes in the real world will will be to the sort of perfect mathematical average that you would estimate just given the chances to begin with. Now, I want to come back to regression towards the mean in um in medical studies because I found a really interesting one that came out earlier this year. Uh So, a lot of a lot of the examples you find involving regression to the mean involved sports or economics, and I found. This one discussed in a New York Times article again from earlier this year titled Intense strength training does not ease knee pain, study finds by Gina Colada. Uh, this is referring to a study published in Jama that entailed an eighteen month clinical trial involving three d and seventy seven participants. Okay, okay, So the basic situation, the setup for this paper is that a lot of people have knee osteoarthritis, and one of the go to treatment recommendations has long been and strength training. So in this study they decided to look into it with three basic groups, one that received intense strength training, another that received moderate strength training, and another that received counseling on healthy living. So that third group, that's the control group, they did not have any amount of strength training, just uh, you know, some positive counseling about healthy living. Sure, so the researchers here apparently actually expected to see the intense strength training take the lead that they were looking to identify what has been just sort of accepted wisdom, um and and again this this has been the predominant treatment idea. But instead they found that the results were the same for all three groups quote, everyone reported slightly less pain, including those who had received only counseling. Now why is that? Well, as Colotta points out, there's there's always room for other effects, especially say the placebo effect. Uh but regression to the is also a heavy consideration here and certainly could work in congress with the placebo effect. Right, So, you don't necessarily have to assume that the counseling actually helped to heal people's knees, though it may have in in in some it may have had some kind of mechanistic effect in some way, a mind body kind of thing. But you would also just expect over time, people who have an extreme starting position, who were starting with a lot of knee pain, to get gradually better over time. Yeah, so a Colatta rights quote are the right As symptoms tend to surge and subside, and people tend to seek out treatments when the pain is at its peak, when it declines, as it would have anyway, they ascribed the improvement to the treatment. Uh. So you know this would this would roughly equate to yelling at your knee when it's in pain, and it really make it certainly relates to many other health scenarios as well various medications and even things like prayer and you know, supernatural um treatments and attempts to to deal with pain, et cetera. Yeah, I mean it could apply to any intervention that is aimed at influencing something that is naturally variable on its own, right. Yeah, and you know something that's again any kind of system in which change occurs when fluctuation occurs. Uh, you know, you can you can see this applying to not only physical pain, but also uh, emotional distress, things of that nature, you know. So again, I think this is an important tool to have in our our logic tool kit. Now there are even cases where I'm tempted to think about the application of regression to the mean, but but where it's probably a lot harder to quantify exactly what the effects are. It's cases where it can be difficult to separate out, say the effects of some kind of deterministic influence like skill versus how how strong the effect of chance or luck is. But I think about things even in the world of the like I think about, you know, the sophomore album by by a band that has like a really stellar debut album. Uh, you know, often that is perceived is disappointing, and you have to wonder, like, Okay, is it is that often true? Because I don't know if people get famous and it goes to their heads and then they you know, they get full of themselves and make something dumb, or is it because when somebody has a debut album that's really well received to some extent, it's so good partially because of luck or chance, and that's an outlier that you're as you're starting sample yeah, yeah, And certainly this is an area that's there's a lot more subjectivity here and and so it's not the kind of thing you can necessarily have a control group for anything. But but I think it is quite interesting. And I did find as I was looking around for some jazzy or examples or possible examples of aggression to the mean, um, I found one that that actually gets into a little bit into the idea of you know, the first and second album. But also uh, the idea of follow up films and Hollywood sequels has pointed out both good Yeah. Has pointed out by Joanna Deong in two thou eighteen on the blogs scientifically sound movie sequels are potentially a great example of aggression to the mean. Quote, Hollywood sequels are only made if the original film is a quote unquote high quality success. But the average quality of sequels will be closer to the mean than average quality of originals of sequels because of regression to the means, So sequels tend to be of lower quality than the original. Now I might somewhat dispute the premise here that Hollywood sequels are only made to films that are high quality to begin with. Um, But but I still think this is onto something because there is a movie that gets a sequel tends to have something about it, something that people are responding to, whether it's a movie that I would like or not. Right, I mean, so sometimes obviously the situation is the film just made a lot of mine. I mean, I guess that's the key thing. It didn't make a lot of money. If so, producers are going to be more inclined to say, let's do that again, Let's have that experience again of all that money coming in. And sometimes this this certainly matches up with a quality film. You have something that really captures people's imagination and it is of high quality and uh and you know, so it's really firing on all cylinders. But you know, and yes, certainly in some cases it's just the right film at the right time. Or or maybe it has nothing to do with the film itself. Maybe it's who's in it, or I don't know what's going on in the zeitgeist during that particular era. Well, the way I would think about this is, and I think again, this is onto something. It highlights that when we experience confusion where we say, like, wow, you know, the Exorcist is such a great horror movie and The Exorcist Too is so bad? How could that be the case? You know, why is it? Why is such a bad sequel to such a great movie? It's because of the compare a son of the original to the sequel that we're experiencing this confusion. Another way you could just look at it is most horror movies are direc most movies are bad, and it is only by comparing the The Exorcist Too to The Exorcist that you notice this steep drop off. Where another way of looking at it is that The Exorcist Too is bad like most horror movies are, and the first one was an outlier. At the beginning, it was a first film in a series that happened to be really good A cut above. Yeah, absolutely, like, yeah, I think this is the correct way to look at it, and also keeping in mind that just how amazing it is that any film gets completed, like even a bad film, Like a lot of people probably worked pretty hard to make that happen, even if the end results don't really please anyone at all. But but yeah, I think this is also an interesting inversion of the opening example of yelling at pilots as well, because most of the time, if a flawed movie comes out, people are not clamoring for the sequel. Um Sequels are rarely guaranteed, so you're not often going to hear things like, oh, well, that wasn't great. I hope the next one is an improvement. I mean some people say that, some people I've said things like that before, where it will be like, oh, really flawed film, but maybe there's like a cool idea I kind of wish it would they would remake it, even though there's no like logical reason that there would be like a there would be money behind that idea. Well, I guess it's kind of different when you're talking about a one off creative project versus something. I mean, we live in a kind of different era now because we were at the height of this you know, cinematic universe thing with a huge number of the big budget movies that come out. The big event movies are not one off creative products, but they are a product that exists within some kind of franchise or universe or something. So you just know automatically that there's gonna be another one, whether this one is good or not. Yeah, like either it's an established film universe where like you know, they put out another Marvel movie and it's just terrible, Well, obviously there's enough momentum. They're not going to stop. They're not gonna be like, oh, well, less and learned, Well we'll stop then. No, No, there's gonna be another. Another example of this might be a successful franchise in another medium, say a book series, so like the Harry Potter books for example, or I don't know, Lord of the Rings, where you know that once they make the Fellowship of the Rings, there's going to be a follow up. They're gonna do another one. So in these ways, unless it's the seventies and it's uh, Lord of the Rings movie that that ends with Helm's Deep. Well, but they picked that up eventually. But kay, but but yeah, probably the Harry Potter films are a better example. And there may be spe specific you know, things about how that wasn't guaranteed either. Uh, you know, the economic reality can always come into play. But for the most part, like those were when when that started, you knew they were gonna keep making these at least they were going to make a follow up, so you could have comments like, well there were that was just on of flawed in some of the some of its execution. I hope that they fix that in the next film. For the most part, yeah, with one offs, this is not the case. It's like, if if this film fizzles, then only you know a few like rare people are going to be clamoring for a sequel or dreaming about what the sequel would be. Yeah, I think this observation, but regression to the mean and movie sequels is actually very on point, but more so for the films of yester Year, where the more the more common thing was you'd have a an independent sort of creative product that it's its own thing, and then if it resonated with somebody, if it did well, there would be sequels. I think it's a little it applies a little bit less today when there's just you know, we're in the world of franchises and extended universes and there's just sort of like a guaranteed, ongoing uh conveyor belt of of new stuff within the Marvel world or the Star Wars world or whatever. Yeah, but I think it it is a worthwhile way to think about creative the creative process, and you know, as opposed to some of these alternate sort of folk wisdomy ways of thinking about it. For example, on Weird House Cinema, we recently talked about Toby Hooper. Toby Hooper is one of those directors who's often you'll often you'll see descriptions. I think we've even read part of a review where they they really they talk about, oh, well, you know he put out Texas Chainsaw Masacre directed that film and this was great. It was, you know, just a real lightning bolt um to the cinematic world into horror itself as a genre. And then the idea that well he was never able to capture that magic again. You know that his his career was just like one long slide after that, which I don't think is a fair assessment, especially if you employ regression to the mean, you know, the idea being that, yeah, he did kind of get lightning in a bottle with that, with that first big film, that that he was able to to really bring something together that is an outlier, um, but that that that's just going to happen. That's just the way these things work, right, So most movies aren't that good, So you of the random chance of like how how good his ideas and execution are from one year to the next is going to set in and you might have a different idea about his career if you were to say, like randomly chronologically reorder all his movies, right, you know, like if you were to put the worst ones earlier on or something, people might feel differently about it. Yeah, well then they would talk about, well, okay, TCM was peak Toby Hooper, like this was his peak output. Because this is the kind of the kind of view of an artist's you know, creative trajectory that we tend to want to um to follow along, you know, because it's more story shaped, the idea of assent and then eventually decent that there's gonna be uh, there's gonna be a period of high noon in their creative out output, and sometimes that does match up with the reality. But I don't know. Even then, we I think we tend to overlook the dogs in the filmographies of people we love, you know. Oh yeah, uh, But then again, I mean, this is interesting because in talking about regression to the mean applying to creative products like movies, we are acknowledging that the creative process is not purely a product of talent and skill, that there is a significant amount of chance and luck involved in something like how good a movie turns out to be? Um, And it's hard to know exactly how to like how to picture that influence of chance and luck, you know, like, what what is that in the creative process? It's obviously true because there are people who can be incredibly skilled in one instance and then I don't know, things just don't go right the next time, and to make something that nobody really likes. But uh, but that's that's just not often how people like to think about creative talents, and people like to think about the creative process like it is much more strictly deterministic. Yeah yeah, or or you look at things like the Star Wars films, and you kind of like fall into this idea of thinking this is stuff that is mind out of the mythic earth, and you know, it just makes sense that things would accumulate and get better. So um, but really looking back on it, especially if you actually like watch documentaries, and there's some great ones about the production of those films, like it's it's amazing that Star Wars, the first one in New Hope was as good as it was, and then it's nothing short of I mean, it's it's just a pure miracle that the second one was so much better and like really nailed it. Like if if the second film had had floundered, I mean, just imagine how different the cinematical landscape would have been for decades to come. Yeah, So it's it's amazing if the first film in a series is good, and it's super amazing if the second one is good. And and this is why I think we often find too that if if part one in part two of something are of high quality, then you've got to look out for that part three because that part three, that part three may be coming to get you. But likewise, if a part two is rubbish, um, you know, subjectively, then then part three might pick it up and uh and get things back on track. So you certainly see that that kind of fluctuation as well. I have a question I actually don't know the answer to, but this would be interesting in terms of I don't know the high performing output, whether that is in whether that is a creative endeavor like you know, writing books or creating movies, or whether that's something even like athletics, like athletic performance, do you expect to see more random fluctuation in the performance of collaborative output versus individual output? So say, um, do you expect more influence of random chance and fluctuation in the quality of uh books written by a single author versus you know, movies that have the input of hundreds of thousands of people? Uh? Or in in the realm of say sports, like do you expect more random variation in the output of an individual athletes like you know, an individual gymnast or something, or in team sports? Yeah? I could see it going both ways, because yeah, if you think too hard to about even just like the film and aology, you can easily get into discussions of like okay, well is it the same cast and crew that are producing the sequel. Uh, you know, what happens when the budget is different, what happens when there are other constraints, what happens when suddenly there are a whole bunch of producers that have their ideas about what things should be. I mean, there's so many different factors to take into place. Uh. You know, with this example that you know, perhaps doesn't bear too close of scrutiny, but but but it's but it's still I think serves as a nice um illustration of the overall trend that we're talking about here. Well, it does bring up the fact that since I mentioned athletes that you know, I don't know a lot about sports. I'm not a big sports fan. But but clearly, but regression to the mean is something that has widely been applied to the world of sports. Uh. For example, in the observation that often after having a really stellar season, either an individual athlete or a sports team will be perceived to underperform the next season. And again, that very well could have something to do with regression to the mean. Like, you know, the fact that they're observed having in a using season is actually an outlier. You're starting your expectations then and saying like, Okay, now they're going to be the best forever. Just by random fluctuation over time, you would expect their next season to probably be not as good as the first. I wonder to what an extent this can be applied to, say, the world of the culinary arts, or even just like various food crops, like say the selecting a cantalope at the grocery store, that sort of thing. I mean, I guess it would apply to pretty much anything where you're sampling in a series over time, there's plenty of random fluctuation in what you're sampling, and the first thing you sample is an outlier in some way really good or really bad. If those things hold true, then you can probably expect you're going to see some regression one way or the other. Yeah. Yeah. By the way, I was looking around for like really stellar examples of a sequel film that is widely believed to be uh rubbish, and I think The Exorcist Too is the primary example. Like you get into some of the other examples that pop up, I feel like there's room for disagreement. Um. For instance, Texas Chainsaw Masacre two is one which I saw popping up on some of these lists for disappointing sequels. But I think that's entirely based on who you ask. I think if you ask us, we will agree that that that t c M Two is is actually a great film. It's different from the first one, perhaps if you go into if you go into part two with the expectations you had for part one, you may see it as a dip in quality. But depending on what else you're bringing to the table, you might see it as an increase in in quality, or at least or something that maybe is different but on par with the original. I mean, it's certainly not for everybody. I mean, it is a It is a gross, disgusting film in in a way like the first one, probably even grosser, but also a sort of satirical masterpiece. Um but I just had another thought when you said that The Exorcist Too is regarded as like one of the best examples of a sequel. That's really rubbish. I mean, it makes me also wonder about the pretty high estimation critics generally have of the exer Is three. Makes me wonder if the effect of The Exorcist to being so bad actually makes people sort of over. You know, they're like they're ready to be impressed by the Exorcist three. Yeah. Yeah, I wonder if that's the case too with it with especially when you have a situation with the part three coming back and restoring uh some I don't know, some level of quality to a franchise. I mean there's also like the Star Trek example, right, I mean that was long the Long held up as an example of like, okay, you have you even Star Treks and your odd Star Treks, right, uh. And I think you've made a similar case for the Faster and Furious movies, right, I mean, once you get to a certain point in the series, I think it's pretty much all uh, you know, a nitrous boosted brain. It's it gets you know, it's all like we're driving cars in space now and flying and all that. But um, but for the earlier ones, yeah, I'd say the odd ones are better. Like, uh, three is the first one where it really starts getting ludicrously weird. Four is kind of a uh and then five starts. Five is when the rock shows up, and then but by seven year golden all right, well we're gonna go ahead and close this one out here. But we'd obviously love to hear from everyone about this about regression towards the mean, just in our daily lives, in various scientific studies. Perhaps you have thoughts about how this applies to something we've discussed on the show in the past, because I know we've we've mentioned regression to the mean in passing before, but certainly we've never taken the opportunity to really dive into it and explain it like we did today. Yeah, I know it's come up in passing, just in us making comments here and there about like the importance of of randomized trials and control groups and all that. In the meantime, if you would like to listen to other episodes of Stuff to Blow Your Mind, you will find them wherever you get your podcast. Just look for the Stuff to Blow your Mind podcast feed. We have our core episodes on Tuesdays and Thursdays, Artifact episodes on Wednesday, listener mail on Monday's. On Fridays, we do a little bit of a weird house cinema. That's our times. We just talk about some sort of a strange film. Uh, and you know, tease apart what makes it strange? Uh, let's see what else so yeah, have you go to stuff to Blow your Mind dot com that will send you to the I Heart listing for our show, and there's a button there for a store if you want to click on that wealth then you can buy some merchandise that has Stuff to Blow your Mind logos and whatnot on it, or perhaps weird House cinema logos and whatnot on it, and you can get mugs, t shirts, all that kind of stuff. Uh, just for fun, you know, or but if you want to be all business about it, you can make it work to get our merch Now, now this this is just for fun, Like, don't use these these clothing items to clothe your nakedness. These are just for fun. This is just purely extra clothing. Okay, this should not be your core clothing. Huge thanks as always to our excellent audio producer Seth Nicholas Johnson. If you would like to get in touch with us with feedback on this episode or any other to suggest a topic for the future, just to say hello, you can email us at contact at stuff to Blow your Mind dot com. Stuff to Blow Your Mind is production of I Heart Radio. For more podcasts for My Heart Radio, visit the I Heart Radio app, Apple podcasts, or wherever you listening to your favorite shows.