The Data Detective

Published Feb 2, 2021, 5:05 AM

Cautionary Tales' host Tim Harford has a new book - The Data Detective - setting out ten commandments for understanding the numbers, charts, graphs and statistics that bombard us every day. In this free extract, Tim explains his extra "golden" rule that allows us to observe all his other commandments - be curious. Enjoy.

Cautionary Tales returns February 26.

The Data Detective (Riverhead Books) is published in the US and Canada on February 2. The same book is available elsewhere under the title How to Make the World Add Up.

(Audio excerpted courtesy of Penguin Random House Audio from The Data Detective by Tim Harford, narrated by the author.)

Learn more about your ad-choices at https://www.iheartpodcastnetwork.com

Pushkin. Hello, Cautionary Tales listeners, Tim Harford here, I have some good news followed by a treat. The good news is that, after long months in the making, the new mega season of Cautionary Tales is about to appear right here on this feed. Fourteen episodes of fiasco and catastrophe, of nerdy insights and heroic failures, and occasionally, not too often, a happy ending. There are murderers, idiots and heroes. There are fraudsters and fighters and whistleblowers. There are gamblers and gamers and geeks galore, all played by a stellar cast of actors, so stellar in fact, that I'm still pinching myself and I'm looking forward to revealing their names very soon. I loved writing this series, and I really hope that you're going to love listening to it, starting weekly on the twenty sixth of February, and now the treat. Loyal listeners may know that my new book, The Data Detective has just been released in the US and Canada. My publishers, Riverhead Books, have kindly agreed to let me share with you the final chapter of the audiobook, in which I reveal the golden rule of thinking about numbers in the news. I've been so pleased with The Data Detective. The international edition was called How to Make the World That Up and was a number one business bestseller in the UK. The Data Detective is a book about how to think clearly about the world by being wiser about statistics and wiser about ourselves and our cognitive biases. In it, I offer ten simple rules to help you be calmer and smarter as you scroll through social media or scan the headlines, and plenty of stories too. The book is available wherever books are sold, and so as the audio book read by yours truly. I hope you like the audiobook extract you're about to hear, and if you do, look out for The Data Detective book, ebook and audiobook, and please spread the word the golden rule be curious. I can think of nothing an audience won't understand. The only problem is to interest them. Once they're interested, they understand anything in the world. Orson wells. I've laid down ten statistical commandments in this book. First, we should learn to stop and notice our emotional reaction to a claim, rather than accepting or rejecting it because of how it makes us feel. Second, we should look for ways to combine the bird's eye statistical perspective with the worm's eye view from personal experience. Third, we should look at the labels on the data we're being given and ask if we understand what's really being described. Fourth, we should look for comparisons and context, putting any claim into perspective. Fifth, we should look behind the statistics, at where they came from and what other data might have vanished into obscurity. Sixth, we should ask who is missing from the data we're being shown, and whether our conclusions might differ if they were included. Seventh, we should ask tough questions about algorithms and the big data sets that drive them, recognizing that without intelligent openness, eight cannot be trusted. Eighth, we should pay more attention to the bedrock of official statistics and the sometimes heroic statisticians who protect it. Ninth we should look under the surface of any beautiful graph or chart, and tenth we should keep an open mind, asking how we might be mistaken and whether the facts have changed. I realize that having ten commandments is something of a cliche, and in truth, they're not commandments so much as rules of thumb or habits of mind that I've acquired the hard way as I've gone along. You might find them worth a try yourself when you come across a statistical claim of particular interest to you. Of course, I don't expect you to run personally through the checklist with every claim you see in the media. Who has the time for that they can be useful in forming a preliminary assessment of your new source? Is the journalist making an effort to define terms, provide context, assess sources. The less these habits of mind are in evidence, the louder alarm bell should ring. Ten rules of thumb is still a lot for anyone to remember, so perhaps I should try to make things simpler. I realize that these suggestions have a common thread, a golden rule. If you like, be curious, look deeper, and ask questions. It is a lot to ask, but I hope that it's not too much. At the start of this book, I begged you not to abandon the idea that we can understand the world by looking at it with the help of statistics. I believe we can and should be able to trust that numbers can give us answers to important questions. But of course nullius inverber we shouldn't trust without also asking questions. The philosopher and Norah O'Neill once declared well placed trust grows out of active inquiry rather than blind acceptance. That seems right. If we want to be able to trust the world around us, we need to show an interest and ask a few basic questions. And despite all the confusions of the modern world, it has never been easier to find answers to those questions. Curiosity, it turns out, can be a remarkably powerful thing. About a decade ago, a Yale University researcher Dan Kahan showed students some footage of a protest outside an unidentified building. Some of the students were told that it was a pro life demonstration outside an abortion clinic. Others were informed that it was a gay rights demonstration outside an army recruitment office. The students were asked some factual questions. Was it a peaceful protest? Did the protesters try to intimidate people passing by? Did they scream or shout? Did they block the entrance to the building. The answers people gave depended on the political identities they embraced. Conservative students who believed they were looking at a demonstration against abortion, saw no problems with a protest, no abuse, no violence, no obstruction. Students on the left who thought they were looking at a gay rights protest reached the same conclusion the protesters had conducted themselves with dignity and restraint. But right wing students who thought they were looking at a gay rights demonstration reached a very different conclusion, as did left wing students who believed they were watching an anti abortion protest. Both these groups concluded that the protesters had been aggressive, intimidating, and obstructive. Kahan was studying a problem we met in the first chapter. The way our political and cultural identity are desired to belong to a community of like minded, right thinking people can, on certain hot button issues, leaders to reach the conclusions we wished to reach. Depressingly, not only do we reach politically comfortable conclusions when parsing complex statistical claims on issues such as climate change, we reach politically comfortable conclusions regardless of the evidence of our own eyes. As we saw earlier, expertise is no guarantee against this kind of motivated reasoning. Republicans and Democrats with high levels of scientific literacy are further apart on climate change than those with little scientific education. The same disheartening pattern holds from nuclear power to gun control to fracking. The more scientifically literate opponents are, the more they disagree. The same is true for numeracy. The greater the proficiency, the more acute the polarization, notes Kahan. After a long and fruitless search for an antidote to tribalism, Kahan could be forgiven for becoming jaded. Yet a few years ago, to his surprise, Kahan and his colleagues stumbled upon a trait that some people have and that other people can be encouraged to develop, which inoculates us against this toxic polarization on the most politically polluted tribal questions, where intelligence and education fail, this trait does not. And if you're desperately, burningly curious to know what it is, congratulations, you may be inoculated already. Curiosity breaks the relentless pattern. Specifically, Kahan identified scientific curiosity that's different from scientific literacy. The two qualities are correlated, of course, but there are curious people who know rather little about science yet and highly trained people with little appetite to learn more. More scientifically curious republic plans aren't further apart from Democrats on these polarized issues. If anything, they're slightly closer together. It's important not to exaggerate the effect. Curious Republicans and Democrats still disagree on issues such as climate change, but the more curious they are, the more they converge on what we might call an evidence based view of the issues in question. Or to put it another way, the more curious we are, the less our tribalism seems to matter. There is little correlation between scientific curiosity and political affiliation. Happily, there are plenty of curious people across the political spectrum. Although the discovery surprised Kahan, it makes sense, as we've seen one of our most stubborn defenses against changing our minds is that we're good at filtering out or dismissing unwelcome information. A curious person, however, enjoys being surprised and hungers for the unexpected. He or she will not be filtering out surprising news because it's far too intriguing. The scientifically curious people Kahan's team studded were originally identified with simple questions buried in a marketing survey so that people weren't conscious that their curiosity was being measured. One question, for example, was how often do you read science books? Scientifically curious people are more interested in watching a documentary about space travel or penguins than a basketball game or a celebrity gossip show. And they didn't just answer survey questions differently, they also made different choices in the psychology lab. In one experiment, participants were shown a range of headlines about climate change and invited to pick the most interesting article to read. There were four headlines, Two suggested climate skepticism and two did not, two reframed as surprising, and two were not. One scientists find still more evidence that global warming actually slowed in last decade skeptical, unsurprising. Two scientists report surprising evidence Arctic ice melting even faster than expected, surprising and not skeptical. Three scientists report surprising evidence ice increasing in Antarctic not currently contributing to sea level rise, skeptical and surprising. Four scientists find still more evidence linking global warming to extreme weather, neither surprising nor skeptical. Typically, we'd expect people to reach for the article that pandered to their prejudices. The Democrats would tend to favor a headline that took global warming seriously, while Republicans would prefer something with a skeptical tone. Scientifically curious people Republicans or Democrats were different. They were happy to grab an article which ran counter to their preconceptions as long as it seemed surprising and fresh, and once you're actually reading the article, there's always a chance that it might teach you something. A surprising statistical claim is a challenge to our existing worldview. It may provoke an emotional response, even a fearful one. Neuroscientific studies suggest that the brain responds in much the same anxious way to facts which threaten our preconceptions as it does to wild animals which threaten our lives. Yet, for someone in a curious frame of mind, in contrast, a surprising claim need not provoke anxiety. It can be an engaging mystery or a puzzle to solve. You're listening to an excerpt of The Data Detective courtesy of Penguin Random House Audio. The Data Detective is a brand new book written and narrated by me, Tim Harford, and we'll be back with more after this message. A curious person might at this point have some questions. When I met Dan Kahan, the question that was most urgent in my mind was can we cultivate curiosity? Can we become more curious? And can we inspire curiosity in others? There are reasons to believe that the answers are yes. One reason, says Kahan, is that his measure of curiosity suggests that incremental change is possible. When he measures scientific curiosity, he doesn't find a lump of stubbornly incurious people at one end of the spectrum and a lump of voraciously curious people at the other, with a yawning gap in the middle. Instead, curiosity follows a continuous bell curve. Most people are either moderately incurious or moderately curious. This doesn't prove that curiosity can be cultivated. Perhaps that bell curve is cast in iron. Yet it does at least hold out some hope that people can be nudged a little further towards the curious end of that curve, because no radical leap is required. A second reason is that curiosity is often situational. In the right place at the right time, curiosity will smolder in any of us. Indeed, Cahan's discovery that an individual's scientific curiosity persisted over time was a surprise to some psychologists. They had believed with some and that there was no such thing as a curious person, just a situation that inspired curiosity. In fact, it does now seem that people can tend to be curious or incurious. That does not alter the fact that curiosity can be fueled or dampened by context. We all have it in us to be curious or not about different things at different times. One thing that provokes curiosity is the sense of a gap in our knowledge to be filled. George Lowenstein, a behavioral economist, framed this idea in what has become known as the information gap theory of curiosity. As Lowenstein puts it, curiosity starts to glow when there's a gap between what we know and what we want to know. There's a sweet spot for curiosity. If we know nothing, we ask no questions. If we know everything, we ask no questions. Either. Curiosity is fueled once we know enough to know that we do not know alas all too often we don't even think about what we don't know. There's a beautiful little experiment about our Incuriosity, conducted by the psychologists Leonard Rosenblitt and Frank Kyle. They gave their experimental subjects a simple task to look through a list of everyday objects, such as a flush lavatory, a zip fastener, and a bicycle, and to rate their understanding of each object on a scale of one to seven. After people had written down their ratings, the researchers would gently launch a devastating ambush. They asked the subjects to elaborate. Here's a pen and paper. They would say, please write out your explanation of a flush lavatory in as much detail as he can by all means include diagrams. It turns out that this task wasn't as easy as people had thought. People stumbled struggling to explain the details of everyday mechanisms. They had assumed that those details would readily spring to mind, and they did not. And to their credit, most experimental subjects realized that they've been lying to themselves. They had felt they understood zip fastness and lavatories, but when invited to elaborate, they realized they didn't understand at all. When people were asked to reconsider their previous one to seven rating, they marked themselves down, acknowledging that their knowledge had been shallower than they'd realized. Rosen Blitt and Kyle called this the illusion of explanatory depth. The illusion of explanatory depth is a curiosity killer and a trap. If we think we already understand, why go deeper? Why ask questions? It is striking that it was so easy to get people to pull back from their earlier confidence. All it took was to get them to reflect on the gaps in their knowledge, and, as Lowenstein argued, gaps in knowledge fuel curiosity. There is more at stake here than zip fastness. Another team of researchers, led by Philip Fernbach and Steve Sloman, authors of The Knowledge Illusion, adapted the flush laboratory question to ask about policies such as a cap and trade system for carbon emissions, a flat tax, or a proposal to impose unilateral sanctions on Iran. The researchers importantly didn't ask people whether or not they were in favor of or against these policies. There's plenty of prior evidence that such questions would lead people to dig in. Instead, Fernbach and his colleagues just ask them the same simple question, Please, rate your understanding on a scale of one to seven. Then the same devastating follow up, please elaborate, tell us exactly what unilateral sanctions are and how a flat tax works, and the same thing happened. People said, yes, they basically understood these policies fairly well. Then when prompted to explain, the illusion was dispelled, they realized that perhaps they didn't really understand at all. More striking was that when the illusion faded, political polarization also started to fade. People who would have instinctively described their political opponents as wicked and who would have gone to the barricades to defend their own ideas tended to be less strident when forced to admit to themselves that they didn't fully understand what it was that they were so passionate about in the first place. The experiment influenced actions as well as words. Search has found that people became less likely to give money to lobby groups or other organizations which supported the positions they had once favored. It's a rather beautiful discovery in a world where so many people seem to hold extreme views with strident certainty. You can deflate somebody's over confidence and moderate their politics simply by asking them to explain the details. Next time you're in a politically heated argument, try asking your interlocutor not to justify herself, but simply to explain the policy in question. She wants to introduce a universal basic income, or a flat tax, or a points based immigration system or medicare for all. Okay, that's interesting. So what exactly does she mean by that? She may learn something as she tries to explain. So may you and you may both find that you understand a little less and agree a little more than you had assumed. Figuring out the workings of a flush lavatory, or understanding what a cap and trade scheme really is, can require some effort. One way to encourage that effort is to embarrass somebody by innocently inviting an overconfident answer on a scale of one to seven. But another kinder way is to engage their interest. As Orson Wells said, once people are interested, they can understand anything in the world. How to engage people's interest is neither a new problem nor an intractable one. Novelists, screenwriters, and comedians have been figuring out this craft for as long as they've existed. They know that we love mysteries, are drawn in by sympathetic characters, enjoy the arc of a good story, and will stick around for anything that makes us laugh, and scientific evidence suggests that Orson Wells was absolutely right. For example, studies in which people were asked to read narratives and non narrative texts found that they zipped through the narrative at twice the speed and recalled twice as much information later. As for humor, consider the case of the comedian Stephen Colbert's civics lesson. Before his current role as the host of The Late Show, Colbert presented The Colbert Report in character as a blowhard right wing commentator. In March twenty eleven, Colbert began a long running joke in which he explored the role of money in US politics. He decided that he needed to set up a political action committee a pack to raise funds in case he decided to run for president. I clearly need a pack, but I have no idea what packs do, he explained to a friendly expert on air. Over the course of the next few weeks, Colbert had packs and super PACs and five or one se fours explained to him from where they could accept donations up to what limits, with what transparency requirements, and to spend on what He was to discover that the right combination of fundraising structures could be used to raise almost any amount of money for almost any purpose with almost no disclosure. Clearly, Sea fours have created an unprecedented, unaccountable, untraceable cash tsunami that will infect every corner of the next election, he mused, and I feel like an idiot for not having one. Colbert later learned how to dissolve his fundraising structures and keep the money without notifying the taxman by repeatedly returning to the topic and in character demanding advice as to how to abuse the electoral rules. Colbert explored campaign finance in far more depth than any news report could have dreamed of doing. Did all of this actually improve viewers knowledge of the issue, It seems so. A team including Kathleen Hall Jamieson, who also worked with Dan Kohan on the Scientific Curiosity Research, used the Colbert storyline to investigate how much people learned amid the laughter. They found that watching the Colbert report was correlated with increased knowledge about super pacts and five O, one C four groups. How they worked what they could legally do. Reading a newspaper or listening to talk radio also helped, but the effect of the Colbert Report was much bigger. One day a week of watching Colbert taught people as much about campaign finance as four days a week reading a newspaper, for example, or five extra years of schooling. Of course, this is a measure of correlation, not causation. It's possible that the people who were already interested in super PACs tuned in to Colbert to hear him wisecrack about them, or perhaps politics junkies know about super PACs and also love watching Colbert. But I suspect the show did cause the growing understanding because Colbert really did go deep into the details, and large audiences stuck with him because he was funny. You don't have to be one of America's best loved comedians to pull off this trick. The NPR podcast Planet Money Wants shed light on the details of the global economy by designing, manufacturing, and importing several thousand T shirts. This allowed a long running storyline investigating cotton farming the role of automation in textiles, how African communities make new fashion out of donated American T shirts, the logistics of the shipping industry, and strange details such as the fact that the men shirts which were made in Bangladesh attract a tariff of sixteen point five percent, whereas the women shirts made in Columbia are duty free. These examples should be models for communication precisely because they inspire curiosity. How does money influence politics is not an especially engaging question, But if I were running for president, how would I raise lots of money with few conditions and no scrutiny is much more intriguing. Those of us in the business of communicating ideas need to go beyond the fact check and the statistical SmackDown. Facts are valuable things, and so is fact checking. But if we really want people to understand complex issues, we need to engage their curiosity. If people are curious, they will learn. I found this in my own work with a team who make more or less for the BBC. The program is often regarded affectionately as a MythBuster. I feel that our best work is when we use statistics to illuminate the truth, rather than to debunker stream of falsehoods. We try to bring people along with us as we explore the world around us with the help of reliable numbers. What's false is interesting, but not as interesting as what's true. After the referendum of twenty sixteen, in which my fellow British voters decided to leave the European Union, the economics profession engaged in some soul searching. Most technical experts thought that leaving the EU was a bad idea, costly complex, and unlikely to deliver many of the promised benefits or solve the country's most pressing problems. Yet, as one infamous sound bite put it, the people in this country have had enough of experts. Few people seemed to care what economists had to say on the subject, and to our credit, I think professional economists wanted to understand what we had done wrong and whether we might do better in future. Later, at a conference about the profession and the Public, the Grades and the Good of the British economics community pondered the problem. The discussed solutions. We needed to be more chatty and approachable on Twitter, suggested one analysis. We needed to express ourselves clearly and without jargon, offered many speakers, not unreasonably. My own perspective was slightly different. I argued that we were operating in a politically polarized environment in which almost any opinion we might offer would be fiercely contested by partisans. Economists deal with controversial issues such as inequality, taxation, public spending, climate change, trade, immigration, and of course Brexit. In such a febrile environment, Speaking slowly and clearly will only get you so far. To communicate complex ideas, we needed to spark people's curiosity, even inspire a sense of wander the great science communicators, after all, people such as Stephen Hawking and David Attenborough do not win over people simply by using small words, crisply spoken. They stoke the flames of our curiosity, making us burn with desire to learn more. If we economists want people to understand economics, we must first engage their interest. What is true of economists is equally true for scientists, social scientists, historians, statisticians, or anyone else with complex ideas to convey. Whether the topic is the evolution of black holes or the emergence of black lives matter, the possibility of precognition or the necessity of preregistration. The details matter, and presented in the right way, they should always have the capacity to fascinate us awaken our sense of wander. I say to my fellow nerd communicators, ignite the spark of curiosity and give it some fuel using the time honored methods of storytelling, character, suspense, and humor. But let's not rely on the journalists and the scientists and the other communicators of complex ideas. We have to be responsible for our own sense of curiosity. As the saying goes, only boring people get bored. The world is so much more interesting if we take an active interest in it. The cure for boredom is curiosity, goes an old saying, there is no cure for curiosity. Just so once we start to peer beneath the surface of things, become aware of the gaps in our knowledge, and treat each question as the path to a better question, we find that curiosity is habit forming. Sometimes we need to think like Darrel Half. There is a place in life for the mean minded, hard nosed skepticism that asks where's the trick? Why is this line bastard lying to me? But while I don't believe it is sometimes the right starting point. When confronted with a surprising statistical claim, it is a lazy and depressing place to finish, and I hope you won't finish there. I hope that I have persuaded you that we should make more room both for the novelty seeking curiosity that says, tell me more, and the dogged curiosity that drove Austin Bradford Hill and Richard Doll to ask why so many people were dying of lung cancer and whether cigarettes might be to blame. If we want to make the world add up, we need to ask questions, open minded, genuine questions, and once we start asking them, we may find it delightfully difficult to stop. That was an extract from my new book, The Data Detective. The International edition is How to make the World Add Up. Thanks for listening, and keep on listening, because Cautionary Tails is back on the twenty sixth of February. Wouldn't be a b

Cautionary Tales with Tim Harford

We tell our children unsettling fairy tales to teach them valuable life lessons, but these Cautionar 
Social links
Follow podcast
Recent clips
Browse 141 clip(s)