Pushkin, what's your biggest fear. I'd say the biggest fear is something a mistake that I would make that would damage my credibility to where people would not listen to me when there's a tornado down. James Span meteorologist, maybe Alabama's best known person aside from some football coaches. He's all over TV talking about the weather, especially when the weather might kill you. So this is a tornado emergency, but the city's at Tuscaloosa and North Fort and the campus of the University of Alabama. James is one of those people who's never really had a job because he found his calling. He once stayed on the air as he watched a tornado make straight for his own home, pleading with people to see the risk. If you're just joining us, This is James Span with Taylor Serrallo mainly chicken on my wife's she's okay and she's in the tornado shelter. Okay, go ahead, Taylor. I'm sorry. I was put on this planet to mitigate loss of life when their tornadoes flying around here, and I have to be very careful in what I say and what I do, not just on the air, but on social media. And in real life. To build trust with his audience, James goes to incredible links. He's published a children's book called Benny and Chipper Prepared Not Scared. He spends time in dollar stores talking to people because the people who shop in dollar stores are also the people who live in trailer homes, the sort of homes that tornadoes obliterate. He memorizes the names of Alabamians who've died in storms, people he might have saved. There's lots of them. On a single day back in April twenty eleven, a line of tornadoes in Alabama killed two hundred and fifty three people. I know their stories, I know their family members. I've talked to many of them, and it's very motivating for me. And that's my main job in life. It's to make the warning process better with severe weather. He's doing all he can to warn people, yet people still don't understand what he's saying. I'm Michael Lewis. Welcome back to Against the Rules, where we explore unfairness in American life by looking at what's happened to various characters in American life. This season is all about experts. Today, we're going to explore the strange thing that's happened to experts. Not all experts, a certain percentage of them, the experts who think and speak in probabilities, who use data to forecast the likelihood of this or that coming to pass. The experts who can never be perfectly certain, and who risk our wrath because we love thinking in absolutes. James Span has been making and explaining weather forecast for the better part of half a century. In that time, it's kind of incredible how much has changed. So here's in nineteen seventy eight forecast partly sunny tomorrow with a chance of showers in the high of eighty. That's it. So today, under the same circumstances, I'd say we'll have a pretty good bit of sunshine between nine and eleven o'clock. After eleven o'clock rain is likely between eleven and one. The chance of any one spot getting wet during that two hour window it's about seventy five percent. It's going to rain about a half inch in most places. There could be some thunder. Most of that should be out of here by two thirty. After three o'clock, you're good to go. The sun breaks back out a temperature should peek around eighty at one o'clock, then falling back into the seventies by four o'clock. That's the difference in what we can do now compared to nineteen seventy eight. It's the prince between daylight and darkness. If you go back to the beginning of your career, were you encouraged to speak to the audience that way, like, we don't know that much about this, this could be wrong. Oh no, no, no, they didn't want you to say that. I mean, coodn't you know. Back in the seventies, this was when TV news was coming of age and I witness news, you know, and they wanted to be this godlike figure, you know on television. I was scared to communicate uncertainty because that wasn't encouraged. We were the news, the evening news, the Ron Burgundy newscast. Weather forecasts are inherently uncertain, the where, the when, the how much. With the current data we have, the best you can do is judge the odds. But the odds have gotten much more accurate over time. Back when James span was a young meteorologist, he knew very little but tried to sound like he knew a lot. Now that he knows a lot, he works hard to explain what he doesn't know. You're giving the audience more information and more new, honest information. So it's more demanding on the audience, right it is. And you know I hear this all the time. I just want to know if it's going to rain tomorrow, and they want a yes orno. They want that deterministic forecast, deterministic as imperfectly predictable, which is something the weather still isn't. When James Span started out, the ten day forecast was no better than just guessing. Now it's a lot better. But maybe the most obvious improvement, the one people really should notice, has been in forecasters understanding of the kind of weather that kills people. In nineteen seventy eight, we were using nineteen fifty seven era radar and the old black and white printouts of radar. It looked like somebody barsed on a piece of paper, and so warnings. In nineteen seventy eight, let's say we had a tornado down. We didn't really know where it was, we had an idea, so warnings were issued by an entire county. Tornadoes, even the big tornadoes are small and counties are huge. So here you are warning an entire county to get into your safe place and do something where most people didn't need to do anything. We're today we know literally within maybe a few city blocks of where the tornado is located. Well, so if I'm a consumer of tornado warnings, I get a much more precise warning, and I do I get a more advanced warning. Am I likely to get it? Get more more time to prepare for this thing? Yes? They have. Average lead time here is about twelve to fifteen minutes, and the average lead time back in the seventies was zero to three minutes. So we've come a long way, and we don't use counties anymore. We use small, small, small segments of counties. Geometric shapes, polygons. Anybody that knows James span I've said this over and over. Respect the polygon, and if you're in it, you do something. Respect the polygon. And if you're in the polygon, you respect the polygon. Respect polygon. Every storm today will mean business. Respect A James Spans superfan did a remix of his famous frame a polygon. I love this, of course, but it also raises a question, why respect the Polygon instead of just respect what I say. It's weird. If the James Span back in nineteen seventy eight had been as accurate as James Span is now, he'd have endured hail storms of gratitude, hurricanes of appreciation, tornadoes of awe. But that's not the wather he now lives with. Hello friends, this is James Span. It's time to read some mean tweets. And thanks to all of you for sending in the mean tweets. I really appreciate from from my heart. You cost the people in the state millions of dollars by your boop poor boot forecasts. I woke up today expecting snow. I blame you, James. I got my dogs all excited for nothing. James, either you're the worst meteorologist I've ever layer my eyes on, or you have the worst luck at predicting the weather. I think it's time to step down. Brother. The only difference between James Span and every other meteorologist is that James reads his mean tweets on the air. Just to show you where we stand, my producer called up some weather tweeters. Here's the kinds of things that people have to say weather forecasting is the only job you can have where you can be wrong fifty percent of the time and still make thousands of dollars. If we were wrong fifty percent in our jobs, we probably would be fired. I know nothing about meteorologists, but I know that you know they always wrong. I'm one of those people that actually vainly looks at the weather forecast because nine times attend it's different from the forecast. As technology improves, they don't improve. The continue to be it's smine boggling. You're going to get the hate, not necessarily because of your missed weather forecast, but just because of who you are. You're a you're a weather person, and you know you're a stooge. You're a you don't deserve to be on the planet. You shouldn't be breathing air. People have that attitude towards weather people. Oh listen. So I cut off a basketball game on Christmas Day in twenty fifteen, and we had a tornado coming up on the southwest part of the city here. It could have killed a lot of people, So we had to cover up about twenty twenty five minutes of that game, and nobody lost their life. The warning system worked beautifully but this is Christmas Day. Joy to the world, peace on earth, goodwill toward men. The first email I got, you know what it said. It said you should have been aborted by a coat hanger. So this is the stuff I deal with. I mean, I'm amazed he still goes on the air. His forecast just keep getting better and better, but the job of being a meteorology just keeps getting worse and worse. But I till these young people, you know, you better have a thick skin when you get out of here and you get your first job, because they're going to come after you when you found that first forecast up. Back in an earlier season of this show, I talked about the problem of referees and a strange phenomenon. A lot of refs are getting better at their jobs. They have new tools, they're better train they get better feedbacks, they are less likely to repeat mistakes. I mean, there's just no way that the refs in pro sports are less accurate than they were forty years ago when there was no replay, less training and all the refs got hired from the same old boys club, But they didn't used to need police escorts from the arenas. Now they do. In December of twenty twenty one, tornadoes ripped through Kentucky. Weather experts gave people lots of warning, So this is just an explosive severe weather set up, and that's the outlook that we have heading our way, especially after midnight through about eight o'clock in the morning. We definitely need to stay aware of the weather game. Meteorologists like this guy on WHASTV and Louisville were better than they'd ever been. Make sure you have a way to wake up if a warning is issued like this one that we have. That night in Kentucky, at least seventy seven people were killed, more people than have ever died from a weather event in the state's history. All people had to do to survive was listened to the experts, and still a lot of them didn't. I think that having data is a really recent phenomenon, Rebecca Golden, math professor at George Mason University. We didn't have data about how things were, we didn't record what happened previously. Then it's only really recently that we think maybe our lived experiences could be in part based on something probabilistic, Like a lot of people who are good at math Rebecca noticed the confusion and wrongheadedness of people who weren't. She also noticed that even when statistics and these new big piles of data were properly explained, people didn't really grasp their meaning. People have a hard time being convinced by data. It's just that they don't think that their experiences is in line with that data, and so they dismiss it, or they have other experiences that tell them that there are reasons to be skeptical of the source of that data or the source of the statements that are relying on the data. The problem isn't just in the quality of the information we have access to. It's in the way we make sense of the world. There's a very large segment of the population who are really struggle with basic mathematics. So people are making mistakes because they don't think in probabilities. I think that's right. Rebecca actually helped to start an organization called stats to expose the statistical mistakes made by journalists. She thought that if statistics were conveyed more accurately to the public, the public would see the world more clearly. Eventually, she decided she was wasting her time because there was this bigger problem how people comprehend statistics, even when they're accurate. Why is it that people don't think in probabilities, like the world's probabilistic. Why are our minds so deterministic? It's kind of a philosophical question. I think we're hardwired to believe. I think it helps us make decisions without being stressed about those decisions. It helps us act with certainty and make decisions so we don't hesitate too much and think too hard. In the savannah, we don't say that's probably a lion, right, we just run. But to just run is less and less a viable way to move through the world world because this relatively new thing called data has given us a far shrewder alternative. Everywhere you turn, you find someone analyzing data to generate the same sort of probabilistic understanding of the world that weather people do. I kind of come from this world of like, you know, kind of quantz and like baseball geeks and like poker players. That's Nate Silver. He got swept up in the nineteen nineties by the statistical revolution in baseball. And I think I'm kind of like one of the relatively people who's kind of escaped, so to speak, from that world in the like mainstream society. Back in two thousand and seven, Nate quit forecasting the future of young baseball players. He began to forecast elections instead. It's sort of what you're doing is actually accepting the possibility that maybe you can predict something that's right. But yeah, it's like kind of like saying, hey, look, we built an audience for this in in baseball, and so politics is still in the Stone Age, and so there must be kind of an audience for some politics too. When you turn your attention to politics, at what point are you aware that the expertise in political forecasting is sort of limited, that there's kind of an opportunity. I mean I had an intuition from that from the very beginning in politics, I mean, the campaigns have to be fairly smart and data driven about they were targeting. But like, but the media was all about kind of narratives. It was really quite bad in two thousand and eight. Right. It's really like a bunch of like, you know, old white men getting together and kind of deciding based on, you know, what their friends think, kind of what the narratives should be in the presidential primaries of two thousand and eight, Nate Silver gave an upstart senator named Barack Obama a much better chance than most everyone else did. In the general election. He nailed not just the outcome, but the result in every state, plus the precise number of votes Obama received in the Electoral College. People paid more attention to what Nate had done than how he had done it. He'd simply use polling data rather than his gut or some anecdote about some Iowa farmer. The polling data might not be perfect, but it was better than every other source of information, and they never made outright predictions. He issued political forecasts like weather forecasts, with probabilities attached to them. Going into election day of two thousand and eight, he'd given Obama a ninety point nine percent chance of winning. I mean, the irather thing about it is like like there was always a chance that we would be wrong, you know what I mean, and probably never heard from politics again, potentially. Instead, Nate became basically overnight the country's leading political forecaster because his expert piece was superior to the storytelling it replaced. Nate Silver is his name, fortune telling is his game. He's a celebrity statistician. Please welcome Nate Silver. That's right, Nate Silver's the good will hunting of political prognosticasia. There's a difference between weather forecasting or sports statistics and politics, a difference more of degree than kind, but still a difference. The people who celebrated Nate Silver really really didn't understand how to judge him. His better insights into pulling data had allowed him to see that Obama was basically always doing better than political pundits thought he was doing, but there was still no law that said Obama had to win. Polling data is a bit like the data that card counters get in blackjack. It's a lot better than having no data at all. It helps you to predict what comes next, but even card counters lose lots of hands. And here we go, ladies and gentlemen, welcome to Decision Night in America. Here at NBC's Democracy Point, which brings us to two thou sixteen. Nate Silver now had an enormous following. Once again, the pundits gave Hillary Clinton better odds than the polls. On election day, Nate gave Donald Trump a roughly thirty percent chance of winning at the time that was a radical call. A few traditional pundits thought Trump had that much of a shot. Yeah, I guess question, guys, are we post Nate Silver, are we pulled out? Well? They've been wrong, not only just wrong, they're just they're superfluous. And at the point where they just that's when you kind of begin to realize that, like, the way you define success and the way other people look at your forecast as being successful are very different. And also because it wasn't just that, like we got criticized after twenty sixteen for having quote unquote been wrong, it was also in the roup twentyteen people were actually mad at us for not being confident enough in Clinton's chances. Right, Nate never claimed to have some mystical ability to call a presidential election, and assigning probabilities is not the same as taking sides. Yelling at him for saying that Donald Trump had a thirty percent chance of winning was like being mad at the weather man for saying there was a thirty percent chance of rain and then getting mad all over again after it rains. I'm gonna get myself in a little bit of trouble for saying this, right, But like people like me really care about being right quote unquote for the intrinsic value of like making a good forecast as opposed to like influencing the narrative, if you will, Okay, so I would love because it would educate me. How do you evaluate a probabilistic a forecast? What's the right way for people to judge Nate's Silver Expert. The right way is if you take a whole bunch of forecasts that we've made and look at how they've done collectively. Right, So, let's say you made one hundred forecasts where the favorite had a seventy percent chance of winning. Look at that group of forecasts, and was it true that the favorite actually won about seventy percent at a time? Right? The slip side of that is that like it does mean that, like, you can tell very little from anyone prediction. I mean, unless you're like, unless you're very very close to one hundred zero percent, right, then one prediction alone won't tell you that much. Experts have gotten better, but they've also gotten harder to judge, so hard that you need an expert to judge them. And that's a problem, right, I Mean, who's going to go to the trouble of evaluating hundreds of Nate Silver's forecasts. And while it's true that he's made thousands of election forecasts, he hasn't made thousands of forecasts for presidential elections. Most people don't even think about elections or forecasts or anything else the way Nate Silver does. Most people don't even speak his language. I actually think that the word uncertainty is used in English in a very different way than uncertainty is used in statistics. Rebecca Golden again, So when we talk about uncertainty and statistics, we might say something about a confidence interval, or we might use a pee value. I'm not really sure you want this on your podcast, Like, maybe that's a little bit too technical. It might be better to trying to think of how it might be better to talk about uncertainty for your Well, this is the root of the matter. So, because it's not just my podcast listeners who are cut above the average human beings, it's like, how the American public understands uncertainty? How do you convey it? I think the best way to talk about it is to actually put it with specific numbers, Like instead of talking percentages, let's talk about numbers instead of ten percent, say one in ten, that kind of thing. But there's a much bigger problem behind all this, an emotional problem. It comes from us wanting certainty in situations where certainty just doesn't exist. If the weatherman says is an eighty percent chance of rain and it doesn't rain, the people on the receiving end of the forecast don't say, oh, that was one of the twenty percent of that was one of the times when it wasn't going to right. They say the expert doesn't know what he's talking about. So the inability to think in terms of probabilities also becomes an inability to evaluate the experts. There's a huge amount of inability to evaluate who is an expert, and yeah, it costs lives, it really does. A new kind of expert appears on the scene, an expert who works with these new big piles of data, an expert who thinks improbabilities, an expert who admits to being uncertain. These new experts are clearly better than the experts they replaced, and yet people treat them as if they're worse and neglect their advice, even when their lives depend on it. We who depend on the experts still want them to have a definitive answer. Either it will reign or it won't. Trump either will win or he won't. But that's not the nature of the world we live in, and we're having some trouble accepting that fact. The more you look for it, the more you see this problem. We've been talking about the problem of the experts getting better yet being treated as if they've gotten worse, a problem that leads to a lot of mystifying behavior, like what's gone on in the past two years inside the American healthcare system. I definitely saw a lot of this coming. Alison Fearing is a nurse at Rush Copley Medical Center in Aurora, Illinois. I can't tell you how many times I've had somebody coming and say I have this because I read X, Y and Z on WebMD, so I know that that's what's going on, and it's like, well, there's a lot more that goes into it that we need to work up further, because there are also other things that this could be and we won't know this until we diagnose it with lab work or a cat scan or whatever. Has this been going on your whole career? I would say it's definitely gotten worse over the past five years or so. I think prior to that there was a bit of it, but definitely not to the extent that there is now. Modern medicine is one of the great miracles of our age. If you went to a doctor in the nineteenth century, he was more likely to kill you than cure you. Now he's vastly more likely to cure you, and the odds of that get better with each passing day. Do you think there's anything that, if it happened to me that required me to go to the emergency room, that I'd be better off forty years ago than now. Honestly, no, I really can't think of a single thing, just because we have so much technology. At first glance, she's not really like James Span or Nate Silver. Doctors and nurses don't usually speak in probabilities, but her expertise is essentially probabilistic. Behind her is a world of medical science that's calculating the odds all the time, the odds that you have this disease as opposed to that one, the odds that this treatment will work versus that one. Every year, Gallup publishes polls that show nursing as America's most trusted profession. But every year the number of people who say they trust nurses and doctors, that number keeps falling. Alison sees it in the number of patients who argue with a diagnosis or treatment. I for one, wouldn't like not ever like go to my mechanic with my car and be like, oh, I know it's you know whatever, because I know nothing about cars, Like I know nothing, and so it's just really wild for me to see that in healthcare, because a human body is so significantly more complicated than a car. If I went on Facebook and I said, if you go jump off the Bay Bridge, you'll fly and it'll be the greatest experience of your life, there are a whole lot of people cann go jump off the Bay Bridge. What is it about this kind of information that causes people to respond to it? The answer popped into my head as soon as I'd ask the question, which I realize raises a question about the question. There are exactly zero examples of people jumping off the Bay Bridge and flying to safety because there's no uncertainty involved. However, there are plenty of examples of doctors and nurses being wrong because medical expertise is a series of probabilistic judgments. The experts are using huge piles of data to judge the odds, the odds that the vaccine will make you ill or keep you safe, which brings us to our most recent national crisis. I have been hit while trying to perform a COVID swab on somebody who is very clearly dying and like crashing fast, and we need to do everything fast, and we say what we're doing. Hey, I'm Ali, I'm your nurse. Today, I'm going to be swabbing your nose real quick for a COVID swab and getting batted at. You know, told that this is not COVID, that this is just bronchitis, and just give me treatment for bronchitis. This isn't what's going on. And it's like watching somebody basically breakdown right in front of you and watching them choose to basically not help themselves. You've heard some version of these stories, and you likely have an opinion about them. But what haunts Allison is one particular case. He was a member of our local police apartment and you know worth as a copy I mean, he was like the epitome of health. He had no pre existing issues, nothing else going on, and unfortunately he was unvaccinated and came in very very ill. I took some time to call his wife and explain to her what was going on, what the game plan was, what we knew so far, and the first thing that she had said to me was just so you guys know, he does not want to be intubated. He knows what happens when people go on the ventilator, and he knows that hospitals are killing people with this. So this is a police officer who thinks hospitals are killing people. Yes, I just had to take a step back and be like, if only you could see the tears in your husband's eyes right now and see how absolutely terrified he is right now, and understand that this is not something that we're wanting to do. Alison was faced with a man with a severe case of COVID who had refused the vaccine. Now, the man's wife wouldn't let the hospital improve his odds of living. We had an extra thirty minutes or so before I had to take him up to the ICU, and so I thought, Okay, you know, I know how this is going to go. I've seen how this has gone. Alison's patient wasn't insane, not in the way a person would be if they jumped off the Bay Bridge thinking they were going to fly when there are zero chances of that happening. The patient had refused a vaccine. There are actual true stories of people getting sick from the vaccine, and look at all those unvaccinated people who were totally fine. The wife was refusing to allow the treatment most likely to save his life. Well, there are actual true cases of people being put on ventilators when they'd been better off if they hadn't. In a probabilistic world, improbable things do happen. We hear stories of the unlikely thing coming true or not coming to pass, and they stick in our minds. But so does Alison's story. I went into his room and you know, gound up and everything, and asked, hey, like, we have a few minutes, do you want to try facetiming your family? And so we get his phone out, we FaceTime his wife, and a couple minutes later, she says, hey, do you want to say hi to the boys? The boys want to say hi to you, And she brings on his young children and I mean they were like three and five years old. If I had to guess, in my heart, I knew this might be the very last time that they ever get to see their dad. And they start saying, Daddy, Daddy, we love you, we love you. And then one says, why aren't you saying it back, dad? And I had to pan the camera over to myself and say, oh, no, he's saying it back. You just can't hear him. The machines are really loud in here, but your daddy loves you, and he's saying it back too, I promise. And we got off the phone call and I got him up to icy you and I had to take a good ten minutes to go to the bathroom and just cry what happened to him. He unfortunately passed away a week later. We're not wired to see the odds. We're not wired to accept the expertise that falls out of a giant pile of data, but our minds still long for the simple answer rooted in our personal experience or some story we've heard, even when the simple answer kills us. We don't naturally respect the polygon, but really we should, Really we should. Against the Rules is written and hosted by me Michael Lewis and produced by Catherine Girardo and Lydia Jeancott. Julia Barton is our editor, with additional editing by Audrey Dilling. Beth Johnson is our fact checker, and Mia Lobell executive produces. Our music is by John Evans and Matthias Bossi of Stellwagon Symphonette. We record our show at Berkeley Advanced Media Studios, expertly helmed by tofur Ruth. Thanks also to Jacob Weisberg, Heather Fain, John Snars, Carly Migliori, Christina Sullivan, Nicole Morano, Royston Deserve, Daniella Lacan, Mary Beth Smith, and Jason Gambrell. And an extra special thanks to Sam Sharpel's for letting us use his amazing respect. The Polygon Remix Against the Rules is a production of Pushkin Industries. Keep in touch, sign up for Pushkin's newsletter at pushkin dot fm, or follow at Pushkin Pods. To find more Pushkin podcasts, listen on the iHeartRadio app, Apple Podcasts, or wherever you listen to podcasts,

Pushkin, what's your biggest fear. I'd say the biggest fear is something a mistake that I would make that would damage my credibility to where people would not listen to me when there's a tornado down. James Span meteorologist, maybe Alabama's best known person aside from some football coaches. He's all over TV talking about the weather, especially when the weather might kill you. So this is a tornado emergency, but the city's at Tuscaloosa and North Fort and the campus of the University of Alabama. James is one of those people who's never really had a job because he found his calling. He once stayed on the air as he watched a tornado make straight for his own home, pleading with people to see the risk. If you're just joining us, This is James Span with Taylor Serrallo mainly chicken on my wife's she's okay and she's in the tornado shelter. Okay, go ahead, Taylor. I'm sorry. I was put on this planet to mitigate loss of life when their tornadoes flying around here, and I have to be very careful in what I say and what I do, not just on the air, but on social media. And in real life. To build trust with his audience, James goes to incredible links. He's published a children's book called Benny and Chipper Prepared Not Scared. He spends time in dollar stores talking to people because the people who shop in dollar stores are also the people who live in trailer homes, the sort of homes that tornadoes obliterate. He memorizes the names of Alabamians who've died in storms, people he might have saved. There's lots of them. On a single day back in April twenty eleven, a line of tornadoes in Alabama killed two hundred and fifty three people. I know their stories, I know their family members. I've talked to many of them, and it's very motivating for me. And that's my main job in life. It's to make the warning process better with severe weather. He's doing all he can to warn people, yet people still don't understand what he's saying. I'm Michael Lewis. Welcome back to Against the Rules, where we explore unfairness in American life by looking at what's happened to various characters in American life. This season is all about experts. Today, we're going to explore the strange thing that's happened to experts. Not all experts, a certain percentage of them, the experts who think and speak in probabilities, who use data to forecast the likelihood of this or that coming to pass. The experts who can never be perfectly certain, and who risk our wrath because we love thinking in absolutes. James Span has been making and explaining weather forecast for the better part of half a century. In that time, it's kind of incredible how much has changed. So here's in nineteen seventy eight forecast partly sunny tomorrow with a chance of showers in the high of eighty. That's it. So today, under the same circumstances, I'd say we'll have a pretty good bit of sunshine between nine and eleven o'clock. After eleven o'clock rain is likely between eleven and one. The chance of any one spot getting wet during that two hour window it's about seventy five percent. It's going to rain about a half inch in most places. There could be some thunder. Most of that should be out of here by two thirty. After three o'clock, you're good to go. The sun breaks back out a temperature should peek around eighty at one o'clock, then falling back into the seventies by four o'clock. That's the difference in what we can do now compared to nineteen seventy eight. It's the prince between daylight and darkness. If you go back to the beginning of your career, were you encouraged to speak to the audience that way, like, we don't know that much about this, this could be wrong. Oh no, no, no, they didn't want you to say that. I mean, coodn't you know. Back in the seventies, this was when TV news was coming of age and I witness news, you know, and they wanted to be this godlike figure, you know on television. I was scared to communicate uncertainty because that wasn't encouraged. We were the news, the evening news, the Ron Burgundy newscast. Weather forecasts are inherently uncertain, the where, the when, the how much. With the current data we have, the best you can do is judge the odds. But the odds have gotten much more accurate over time. Back when James span was a young meteorologist, he knew very little but tried to sound like he knew a lot. Now that he knows a lot, he works hard to explain what he doesn't know. You're giving the audience more information and more new, honest information. So it's more demanding on the audience, right it is. And you know I hear this all the time. I just want to know if it's going to rain tomorrow, and they want a yes orno. They want that deterministic forecast, deterministic as imperfectly predictable, which is something the weather still isn't. When James Span started out, the ten day forecast was no better than just guessing. Now it's a lot better. But maybe the most obvious improvement, the one people really should notice, has been in forecasters understanding of the kind of weather that kills people. In nineteen seventy eight, we were using nineteen fifty seven era radar and the old black and white printouts of radar. It looked like somebody barsed on a piece of paper, and so warnings. In nineteen seventy eight, let's say we had a tornado down. We didn't really know where it was, we had an idea, so warnings were issued by an entire county. Tornadoes, even the big tornadoes are small and counties are huge. So here you are warning an entire county to get into your safe place and do something where most people didn't need to do anything. We're today we know literally within maybe a few city blocks of where the tornado is located. Well, so if I'm a consumer of tornado warnings, I get a much more precise warning, and I do I get a more advanced warning. Am I likely to get it? Get more more time to prepare for this thing? Yes? They have. Average lead time here is about twelve to fifteen minutes, and the average lead time back in the seventies was zero to three minutes. So we've come a long way, and we don't use counties anymore. We use small, small, small segments of counties. Geometric shapes, polygons. Anybody that knows James span I've said this over and over. Respect the polygon, and if you're in it, you do something. Respect the polygon. And if you're in the polygon, you respect the polygon. Respect polygon. Every storm today will mean business. Respect A James Spans superfan did a remix of his famous frame a polygon. I love this, of course, but it also raises a question, why respect the Polygon instead of just respect what I say. It's weird. If the James Span back in nineteen seventy eight had been as accurate as James Span is now, he'd have endured hail storms of gratitude, hurricanes of appreciation, tornadoes of awe. But that's not the wather he now lives with. Hello friends, this is James Span. It's time to read some mean tweets. And thanks to all of you for sending in the mean tweets. I really appreciate from from my heart. You cost the people in the state millions of dollars by your boop poor boot forecasts. I woke up today expecting snow. I blame you, James. I got my dogs all excited for nothing. James, either you're the worst meteorologist I've ever layer my eyes on, or you have the worst luck at predicting the weather. I think it's time to step down. Brother. The only difference between James Span and every other meteorologist is that James reads his mean tweets on the air. Just to show you where we stand, my producer called up some weather tweeters. Here's the kinds of things that people have to say weather forecasting is the only job you can have where you can be wrong fifty percent of the time and still make thousands of dollars. If we were wrong fifty percent in our jobs, we probably would be fired. I know nothing about meteorologists, but I know that you know they always wrong. I'm one of those people that actually vainly looks at the weather forecast because nine times attend it's different from the forecast. As technology improves, they don't improve. The continue to be it's smine boggling. You're going to get the hate, not necessarily because of your missed weather forecast, but just because of who you are. You're a you're a weather person, and you know you're a stooge. You're a you don't deserve to be on the planet. You shouldn't be breathing air. People have that attitude towards weather people. Oh listen. So I cut off a basketball game on Christmas Day in twenty fifteen, and we had a tornado coming up on the southwest part of the city here. It could have killed a lot of people, So we had to cover up about twenty twenty five minutes of that game, and nobody lost their life. The warning system worked beautifully but this is Christmas Day. Joy to the world, peace on earth, goodwill toward men. The first email I got, you know what it said. It said you should have been aborted by a coat hanger. So this is the stuff I deal with. I mean, I'm amazed he still goes on the air. His forecast just keep getting better and better, but the job of being a meteorology just keeps getting worse and worse. But I till these young people, you know, you better have a thick skin when you get out of here and you get your first job, because they're going to come after you when you found that first forecast up. Back in an earlier season of this show, I talked about the problem of referees and a strange phenomenon. A lot of refs are getting better at their jobs. They have new tools, they're better train they get better feedbacks, they are less likely to repeat mistakes. I mean, there's just no way that the refs in pro sports are less accurate than they were forty years ago when there was no replay, less training and all the refs got hired from the same old boys club, But they didn't used to need police escorts from the arenas. Now they do. In December of twenty twenty one, tornadoes ripped through Kentucky. Weather experts gave people lots of warning, So this is just an explosive severe weather set up, and that's the outlook that we have heading our way, especially after midnight through about eight o'clock in the morning. We definitely need to stay aware of the weather game. Meteorologists like this guy on WHASTV and Louisville were better than they'd ever been. Make sure you have a way to wake up if a warning is issued like this one that we have. That night in Kentucky, at least seventy seven people were killed, more people than have ever died from a weather event in the state's history. All people had to do to survive was listened to the experts, and still a lot of them didn't. I think that having data is a really recent phenomenon, Rebecca Golden, math professor at George Mason University. We didn't have data about how things were, we didn't record what happened previously. Then it's only really recently that we think maybe our lived experiences could be in part based on something probabilistic, Like a lot of people who are good at math Rebecca noticed the confusion and wrongheadedness of people who weren't. She also noticed that even when statistics and these new big piles of data were properly explained, people didn't really grasp their meaning. People have a hard time being convinced by data. It's just that they don't think that their experiences is in line with that data, and so they dismiss it, or they have other experiences that tell them that there are reasons to be skeptical of the source of that data or the source of the statements that are relying on the data. The problem isn't just in the quality of the information we have access to. It's in the way we make sense of the world. There's a very large segment of the population who are really struggle with basic mathematics. So people are making mistakes because they don't think in probabilities. I think that's right. Rebecca actually helped to start an organization called stats to expose the statistical mistakes made by journalists. She thought that if statistics were conveyed more accurately to the public, the public would see the world more clearly. Eventually, she decided she was wasting her time because there was this bigger problem how people comprehend statistics, even when they're accurate. Why is it that people don't think in probabilities, like the world's probabilistic. Why are our minds so deterministic? It's kind of a philosophical question. I think we're hardwired to believe. I think it helps us make decisions without being stressed about those decisions. It helps us act with certainty and make decisions so we don't hesitate too much and think too hard. In the savannah, we don't say that's probably a lion, right, we just run. But to just run is less and less a viable way to move through the world world because this relatively new thing called data has given us a far shrewder alternative. Everywhere you turn, you find someone analyzing data to generate the same sort of probabilistic understanding of the world that weather people do. I kind of come from this world of like, you know, kind of quantz and like baseball geeks and like poker players. That's Nate Silver. He got swept up in the nineteen nineties by the statistical revolution in baseball. And I think I'm kind of like one of the relatively people who's kind of escaped, so to speak, from that world in the like mainstream society. Back in two thousand and seven, Nate quit forecasting the future of young baseball players. He began to forecast elections instead. It's sort of what you're doing is actually accepting the possibility that maybe you can predict something that's right. But yeah, it's like kind of like saying, hey, look, we built an audience for this in in baseball, and so politics is still in the Stone Age, and so there must be kind of an audience for some politics too. When you turn your attention to politics, at what point are you aware that the expertise in political forecasting is sort of limited, that there's kind of an opportunity. I mean I had an intuition from that from the very beginning in politics, I mean, the campaigns have to be fairly smart and data driven about they were targeting. But like, but the media was all about kind of narratives. It was really quite bad in two thousand and eight. Right. It's really like a bunch of like, you know, old white men getting together and kind of deciding based on, you know, what their friends think, kind of what the narratives should be in the presidential primaries of two thousand and eight, Nate Silver gave an upstart senator named Barack Obama a much better chance than most everyone else did. In the general election. He nailed not just the outcome, but the result in every state, plus the precise number of votes Obama received in the Electoral College. People paid more attention to what Nate had done than how he had done it. He'd simply use polling data rather than his gut or some anecdote about some Iowa farmer. The polling data might not be perfect, but it was better than every other source of information, and they never made outright predictions. He issued political forecasts like weather forecasts, with probabilities attached to them. Going into election day of two thousand and eight, he'd given Obama a ninety point nine percent chance of winning. I mean, the irather thing about it is like like there was always a chance that we would be wrong, you know what I mean, and probably never heard from politics again, potentially. Instead, Nate became basically overnight the country's leading political forecaster because his expert piece was superior to the storytelling it replaced. Nate Silver is his name, fortune telling is his game. He's a celebrity statistician. Please welcome Nate Silver. That's right, Nate Silver's the good will hunting of political prognosticasia. There's a difference between weather forecasting or sports statistics and politics, a difference more of degree than kind, but still a difference. The people who celebrated Nate Silver really really didn't understand how to judge him. His better insights into pulling data had allowed him to see that Obama was basically always doing better than political pundits thought he was doing, but there was still no law that said Obama had to win. Polling data is a bit like the data that card counters get in blackjack. It's a lot better than having no data at all. It helps you to predict what comes next, but even card counters lose lots of hands. And here we go, ladies and gentlemen, welcome to Decision Night in America. Here at NBC's Democracy Point, which brings us to two thou sixteen. Nate Silver now had an enormous following. Once again, the pundits gave Hillary Clinton better odds than the polls. On election day, Nate gave Donald Trump a roughly thirty percent chance of winning at the time that was a radical call. A few traditional pundits thought Trump had that much of a shot. Yeah, I guess question, guys, are we post Nate Silver, are we pulled out? Well? They've been wrong, not only just wrong, they're just they're superfluous. And at the point where they just that's when you kind of begin to realize that, like, the way you define success and the way other people look at your forecast as being successful are very different. And also because it wasn't just that, like we got criticized after twenty sixteen for having quote unquote been wrong, it was also in the roup twentyteen people were actually mad at us for not being confident enough in Clinton's chances. Right, Nate never claimed to have some mystical ability to call a presidential election, and assigning probabilities is not the same as taking sides. Yelling at him for saying that Donald Trump had a thirty percent chance of winning was like being mad at the weather man for saying there was a thirty percent chance of rain and then getting mad all over again after it rains. I'm gonna get myself in a little bit of trouble for saying this, right, But like people like me really care about being right quote unquote for the intrinsic value of like making a good forecast as opposed to like influencing the narrative, if you will, Okay, so I would love because it would educate me. How do you evaluate a probabilistic a forecast? What's the right way for people to judge Nate's Silver Expert. The right way is if you take a whole bunch of forecasts that we've made and look at how they've done collectively. Right, So, let's say you made one hundred forecasts where the favorite had a seventy percent chance of winning. Look at that group of forecasts, and was it true that the favorite actually won about seventy percent at a time? Right? The slip side of that is that like it does mean that, like, you can tell very little from anyone prediction. I mean, unless you're like, unless you're very very close to one hundred zero percent, right, then one prediction alone won't tell you that much. Experts have gotten better, but they've also gotten harder to judge, so hard that you need an expert to judge them. And that's a problem, right, I Mean, who's going to go to the trouble of evaluating hundreds of Nate Silver's forecasts. And while it's true that he's made thousands of election forecasts, he hasn't made thousands of forecasts for presidential elections. Most people don't even think about elections or forecasts or anything else the way Nate Silver does. Most people don't even speak his language. I actually think that the word uncertainty is used in English in a very different way than uncertainty is used in statistics. Rebecca Golden again, So when we talk about uncertainty and statistics, we might say something about a confidence interval, or we might use a pee value. I'm not really sure you want this on your podcast, Like, maybe that's a little bit too technical. It might be better to trying to think of how it might be better to talk about uncertainty for your Well, this is the root of the matter. So, because it's not just my podcast listeners who are cut above the average human beings, it's like, how the American public understands uncertainty? How do you convey it? I think the best way to talk about it is to actually put it with specific numbers, Like instead of talking percentages, let's talk about numbers instead of ten percent, say one in ten, that kind of thing. But there's a much bigger problem behind all this, an emotional problem. It comes from us wanting certainty in situations where certainty just doesn't exist. If the weatherman says is an eighty percent chance of rain and it doesn't rain, the people on the receiving end of the forecast don't say, oh, that was one of the twenty percent of that was one of the times when it wasn't going to right. They say the expert doesn't know what he's talking about. So the inability to think in terms of probabilities also becomes an inability to evaluate the experts. There's a huge amount of inability to evaluate who is an expert, and yeah, it costs lives, it really does. A new kind of expert appears on the scene, an expert who works with these new big piles of data, an expert who thinks improbabilities, an expert who admits to being uncertain. These new experts are clearly better than the experts they replaced, and yet people treat them as if they're worse and neglect their advice, even when their lives depend on it. We who depend on the experts still want them to have a definitive answer. Either it will reign or it won't. Trump either will win or he won't. But that's not the nature of the world we live in, and we're having some trouble accepting that fact. The more you look for it, the more you see this problem. We've been talking about the problem of the experts getting better yet being treated as if they've gotten worse, a problem that leads to a lot of mystifying behavior, like what's gone on in the past two years inside the American healthcare system. I definitely saw a lot of this coming. Alison Fearing is a nurse at Rush Copley Medical Center in Aurora, Illinois. I can't tell you how many times I've had somebody coming and say I have this because I read X, Y and Z on WebMD, so I know that that's what's going on, and it's like, well, there's a lot more that goes into it that we need to work up further, because there are also other things that this could be and we won't know this until we diagnose it with lab work or a cat scan or whatever. Has this been going on your whole career? I would say it's definitely gotten worse over the past five years or so. I think prior to that there was a bit of it, but definitely not to the extent that there is now. Modern medicine is one of the great miracles of our age. If you went to a doctor in the nineteenth century, he was more likely to kill you than cure you. Now he's vastly more likely to cure you, and the odds of that get better with each passing day. Do you think there's anything that, if it happened to me that required me to go to the emergency room, that I'd be better off forty years ago than now. Honestly, no, I really can't think of a single thing, just because we have so much technology. At first glance, she's not really like James Span or Nate Silver. Doctors and nurses don't usually speak in probabilities, but her expertise is essentially probabilistic. Behind her is a world of medical science that's calculating the odds all the time, the odds that you have this disease as opposed to that one, the odds that this treatment will work versus that one. Every year, Gallup publishes polls that show nursing as America's most trusted profession. But every year the number of people who say they trust nurses and doctors, that number keeps falling. Alison sees it in the number of patients who argue with a diagnosis or treatment. I for one, wouldn't like not ever like go to my mechanic with my car and be like, oh, I know it's you know whatever, because I know nothing about cars, Like I know nothing, and so it's just really wild for me to see that in healthcare, because a human body is so significantly more complicated than a car. If I went on Facebook and I said, if you go jump off the Bay Bridge, you'll fly and it'll be the greatest experience of your life, there are a whole lot of people cann go jump off the Bay Bridge. What is it about this kind of information that causes people to respond to it? The answer popped into my head as soon as I'd ask the question, which I realize raises a question about the question. There are exactly zero examples of people jumping off the Bay Bridge and flying to safety because there's no uncertainty involved. However, there are plenty of examples of doctors and nurses being wrong because medical expertise is a series of probabilistic judgments. The experts are using huge piles of data to judge the odds, the odds that the vaccine will make you ill or keep you safe, which brings us to our most recent national crisis. I have been hit while trying to perform a COVID swab on somebody who is very clearly dying and like crashing fast, and we need to do everything fast, and we say what we're doing. Hey, I'm Ali, I'm your nurse. Today, I'm going to be swabbing your nose real quick for a COVID swab and getting batted at. You know, told that this is not COVID, that this is just bronchitis, and just give me treatment for bronchitis. This isn't what's going on. And it's like watching somebody basically breakdown right in front of you and watching them choose to basically not help themselves. You've heard some version of these stories, and you likely have an opinion about them. But what haunts Allison is one particular case. He was a member of our local police apartment and you know worth as a copy I mean, he was like the epitome of health. He had no pre existing issues, nothing else going on, and unfortunately he was unvaccinated and came in very very ill. I took some time to call his wife and explain to her what was going on, what the game plan was, what we knew so far, and the first thing that she had said to me was just so you guys know, he does not want to be intubated. He knows what happens when people go on the ventilator, and he knows that hospitals are killing people with this. So this is a police officer who thinks hospitals are killing people. Yes, I just had to take a step back and be like, if only you could see the tears in your husband's eyes right now and see how absolutely terrified he is right now, and understand that this is not something that we're wanting to do. Alison was faced with a man with a severe case of COVID who had refused the vaccine. Now, the man's wife wouldn't let the hospital improve his odds of living. We had an extra thirty minutes or so before I had to take him up to the ICU, and so I thought, Okay, you know, I know how this is going to go. I've seen how this has gone. Alison's patient wasn't insane, not in the way a person would be if they jumped off the Bay Bridge thinking they were going to fly when there are zero chances of that happening. The patient had refused a vaccine. There are actual true stories of people getting sick from the vaccine, and look at all those unvaccinated people who were totally fine. The wife was refusing to allow the treatment most likely to save his life. Well, there are actual true cases of people being put on ventilators when they'd been better off if they hadn't. In a probabilistic world, improbable things do happen. We hear stories of the unlikely thing coming true or not coming to pass, and they stick in our minds. But so does Alison's story. I went into his room and you know, gound up and everything, and asked, hey, like, we have a few minutes, do you want to try facetiming your family? And so we get his phone out, we FaceTime his wife, and a couple minutes later, she says, hey, do you want to say hi to the boys? The boys want to say hi to you, And she brings on his young children and I mean they were like three and five years old. If I had to guess, in my heart, I knew this might be the very last time that they ever get to see their dad. And they start saying, Daddy, Daddy, we love you, we love you. And then one says, why aren't you saying it back, dad? And I had to pan the camera over to myself and say, oh, no, he's saying it back. You just can't hear him. The machines are really loud in here, but your daddy loves you, and he's saying it back too, I promise. And we got off the phone call and I got him up to icy you and I had to take a good ten minutes to go to the bathroom and just cry what happened to him. He unfortunately passed away a week later. We're not wired to see the odds. We're not wired to accept the expertise that falls out of a giant pile of data, but our minds still long for the simple answer rooted in our personal experience or some story we've heard, even when the simple answer kills us. We don't naturally respect the polygon, but really we should, Really we should. Against the Rules is written and hosted by me Michael Lewis and produced by Catherine Girardo and Lydia Jeancott. Julia Barton is our editor, with additional editing by Audrey Dilling. Beth Johnson is our fact checker, and Mia Lobell executive produces. Our music is by John Evans and Matthias Bossi of Stellwagon Symphonette. We record our show at Berkeley Advanced Media Studios, expertly helmed by tofur Ruth. Thanks also to Jacob Weisberg, Heather Fain, John Snars, Carly Migliori, Christina Sullivan, Nicole Morano, Royston Deserve, Daniella Lacan, Mary Beth Smith, and Jason Gambrell. And an extra special thanks to Sam Sharpel's for letting us use his amazing respect. The Polygon Remix Against the Rules is a production of Pushkin Industries. Keep in touch, sign up for Pushkin's newsletter at pushkin dot fm, or follow at Pushkin Pods. To find more Pushkin podcasts, listen on the iHeartRadio app, Apple Podcasts, or wherever you listen to podcasts,