Puzzle Rush

Published Jun 20, 2019, 9:00 AM

Malcolm challenges his assistant Camille to the Law School Admissions Test. He gets halfway through, panics, runs out of time, and wonders: why does the legal world want him to rush?

Learn more about your ad-choices at https://www.iheartpodcastnetwork.com

Pushkin. It's eight o'clock in the morning on a Saturday, a little chilly deep in downtown Manhattan. Streets are empty. I'm standing on a sidewalk with my assistant, Camille Baptista. First of all, Camille, UM, I need to know did you did you sleep well last? Then? I did sleep well? Yes, I did not, really, I could almost no sleep. And I had a nightmare about the l said that I laughed before himself. I had an exam nightmare from high school, essentially a high school, and you laughed before it was over. I had a nightmare about the el sad that I walked out before the test was over. Oh but my nightmare was just beginning. This is Malcolm Gladwell. I'm back with season four of Revisionist History, my podcast about things overlooked and misunderstood. This episode is about what happened when Camille and I took the law school admissions test. We had three number two pencils each, a small pack of Kleenex, a package of trail Mix, all in clear plastic bags. We lined up outside Pace University with hundreds of other nervous people clutching clear plastic bags. Everyone wanted someday to become lawyers, except us. We were there in the name of science. We had to go. Yeah, it's all been very Uh it's traumatic and stressful, but I wanted to get one last thing. I would like to get your handicap on your chances of beating Just tell me what you think your chances of beating me here, Um, my chances as as your mom point out, was quite confident you would win. My mom is confident me no matter what, which is very nice, but you know, maybe unrealistic. Uh, you know, I don't know fifty fifty Camille, I really, I told you I am twenty four. Okay, I can't even I won't even tell you how old I am. But it's it is a It is a large multiple of your age, I know. But I don't think it has I don't think it has to do with age. I think it has to do with reading ability. And you read NonStop twenty four hours a day. I've been an intellectual. I've been in cognitive decline for years. It's all going to be exposed. All right, we're off. I got the idea of taking the l SAT from a man named William Henderson. I read a paper that he wrote It was on my favorite website SSRN, which is where academics from around the world posts their papers and they get ranked. If you're a regular listener to this podcast, you'll know how genius I think SSRN is. Anyway, this paper was called the l s at Law School Exams and Meritocracy, The Surprising and under theorized role of test taking speed. Anything with the words surprising and under theorized in the title, of course, is going to be catnet for me. So I read it, then I read it again. Then I had to meet him. So just a little bit of background here. I didn't go to law school to US thirty five, and so I had had a whole career before then. Why did What were you doing before you became a lawyer? I was a firefighter paramedic. I was a union rep for a suburban Cleveland fire department. I did that for nine years. Is a kid, people, And remember I'm kidding, as I went to law school because of those Mastard management attorneys. Henderson goes to law school. After that, he gets a clerkship with a judge and one day he's in the shower and a thought occurs to him, what was the aha in a shower. The specific aha was what was my God? The two most time pressure things I've ever done in my life is taking the LSAT in these damn law school exams. He had been a firefighter paramedic, he had raced to the streets of Cleveland to save people on the brink of death. And then most time pressure he'd ever felt was taking the l set together into law school and then taking the exams once he got there. I can remember taking Cass Sunstein's Elements exam my first year of law school is a two hour exam. Cass sun used to teach at the University of Chicago, which is where Henderson went to law school. He's a genius. He asked the kind of questions that you can spend months years thinking about. But his exam two hours. That's all you got. The proctor said time, and everybody dropped their pencils and there is a huge ug. You know, everybody was like groaning that they didn't get more time to keep on working on Cash Sunstein's element exam. Just like you know, ninety people who all drop their pencil and they're all, you know, exclaiming the desire for more time. That was a memorable experience. Now, I'm sure this is obvious to you, particularly if you're an American. You've been tested a thousand times in your life, SAT ACT, GRE, on and on, and every one of those standardized tests doesn't just test whether you can answer the question correctly. They test how quickly you can answer the question correctly. But I'm a Canadian. I've never taken a standardized test in my life, and as an outsider, I have to say the whole system seems really weird. Why is quicker better? V L set is Exhibit A. It's the single most important thing that determines where you go to law school. Nothing else comes close. And what is it? It's around one hundred and twenty five questions, divided among five sections analytical reasoning, reading, comprehension, two sections on dissecting an argument, and one experimental section. Everything's multiple choice. You mark your answers with a pencil by filling in little bubbles on an answer sheet like an IBM punchcard. A perfect score is one eighty, and if you're a super go getter, you cannot score below one seventy five because then you can't get into Harvard. And if you can't get into Harvard. You're never going to get an offer from a big law firm or get a Supreme Court clerkship. Your life is over everything. The country club, the BMW, the multiple cases of two thousand, Chateau Lafitte, rothschild, it hinges on those five sections, and how long do you get to spend on each of those five sections? Thirty five minutes, not forty, not fifty, not fifty five. And if you finish up one section early, you don't get to add that time to the next section. When we decide who is smart enough to be a lawyer, we use a stopwatch. So that was William Henderson's question. What does putting that thirty five minute time limit on a cognitive test like the els AT do? So I said to Camille, let's find out, and because we're both super competitive, the whole thing got very involved. The first thing I would like to do is to understand on a very very granular level, what time pressure means for the way people take the test, the strategies they use, the kinds of mistakes they may specific Everyone told me that there was no way I could do well on the l set without getting some coaching. So I went to one of the top new educational technology startups in America, a company in New York called Noodle, started by John Katzman, who before this was one of the founders of the famous Princeton Review test prep company. I sat down with Katsman and two of his top people in a big bluff building on Union Square in Manhattan, Catsman Fritz Stewart and Dan Edmonds. No, but I am very, very anxious not to lose to Camille. So I thought, was she born in the US or in the U. Oh, she's went to a fancy school like this serious, seriously intelligent woman. But I'm saying US, UK were US. Okay, you have a problem. Oh great, I'm too old and I'm from the wrong country. It was not an encouraging start. I've done things under time pressure before. Of course, I've written newspaper stories on deadline. I wrote exams in college. In those cases, though, the task required me to be me, only with a sense of urgency. But the first thing the Noodle guys told me was the l SAT didn't require me to be me. It required me to be someone else. A lot of it is about helping people understand they don't get to do this at a comfortable speed. That's Dan Edmonds right. In order to finish these sections, you have to do it at a speed that's a little uncomfortable, which means you have to hone your instincts by the test rules, not by your rules. What does it mean to read it at an uncomfortable speed? For me? It means that if there's something like in a reading comprehension passage, if I hit a paragraph that I don't fully understand, I don't get to go back and reread it. I just kind of have to accept the parts that I understand and move on. The noodle guys approached the l set like pathologists approach a cadaver. For me, I'm like, all right, I didn't get that, but if a question asks about it, I will go back. But it doesn't bother me as long as I understand the topic, sentence of the paragraph, and the overall thrust of the author's argument. If I miss a few details here and there, or even a chunk of the argument, I'm all right, fine, who cares, I'll go. I'll go dig it out exactly what I need to. Wait, it's okay to miss a chunk of the argument, So I don't even aim for a level of what we would normally call comprehension, and my first read I am out to process the information, not understand it. I don't get any points for understanding it. I get points for bubbling in the right question. I imagine that when you read, there are lots of moments where you sort of oh, that's an interesting point, and you sort of pause and you think, and you let your mind meander a little bit. There's no meander time. On the off side, there's no digression time. Meander and digression are my whole m. That's what I do for a living. What do you think this podcast comes from? So then John Katzman gave me a sample question. What I would have you do is read this, and then for each of these questions, tell me the two stupidest answers, two stupidest okay, the two answers choices that you know are wrong. WHOA, this is what? That's what I'm saying. You can't read this on a podcast. I don't know. I can't the passage with six hundred words it seemed longer. I've just been told that I don't need to understand it. I don't need to comprehend it. It's okay if I miss chunks of it. In other words, I'm supposed to read it without reading it. And as I sit there puzzling about this, the three of them debate right in front of me. How long is reading without reading should take? John said a minute, Fritz said three tops, Dan said it depends on the person who's reading. The only thing they all agreed on was that I had to hurry fly, get the bones of the argument, and now feel free to spend the time on the question itself. And again, all you're trying to do is tell me the two worst answers. Two worst answers. When do you tell me? I'm on your shop? Just so? He is on page one forty two? Right, go okay? I read it out loud because I thought maybe that would help researcher. People who participate in opinion surveys often give answers they believe the opinion surveyor wants to hear, and for this reason that some opinion surveys do not reflect the actual views of those being surveyed. However, in well constructed surveys, the questions are worded so as to provide respondence with no indication of which answers the surveyor might expect after the passage, there were multiple sets of questions to test my understanding of what I had just not read. They were pages of them. So if a survey is well constructed, survey respondence desire to meet surveyor's expectations has no effect on the survey's results. The reasoning in the researchers argument is questionable in that the argument overlooks the possibility that they say A and B are just just a cross map and move on from actions. Okay, those are the worst. Not they're wrong, They're the worst answers. The clock is ticking. This is such a brutal passage, shining cruel. Jesus bananas. What am I aunt? Time wise? This is what they meant when they talked about uncomfortable reading. I was being forced into a kind of altered, frenzied state. The word they used was breathless. You should be a little breathless, they said. I was breathless. I kept asking how much time has passed? They tell me my heart would accelerate. I started to panic. Okay, okay, I have no idea what I did great timing on that seven minute's twenty. So this section is how long a few minutes? Thirty five minutes, thirty five, thirty five minutes. And we just did one, two, three, four, five, six seven questions you would have had, say, ten minutes. Yeah, but I haven't picked any right answers yet. Well, you've done whole the Harper and that was four and a half seven twenty. So you're tight. Oh man, you're not toast. Oh I'm toast. We did another question. I was convinced the answer was B. It was C. I could feel Harvard slipping away. Great, thank you. What will you guys think my aunts are of the eating commute? If you want the honest answer, I think you're in real trouble about you. She's got On the day of the l set, I sat in a little class room in one of those fixed half desks that I last had in grade school. We got our test packets. I raced through reading comprehension with time to spare sections two and three. I was right up against the deadline. Then came analytical reasoning logic games, and I looked at the questions and what I had to figure out and realized that there was just no way I was going to finish in thirty five minutes. I needed to slow down, and I wasn't allowed to slow down. I glanced behind me and there was Camille with like a death stare in her eyes, ruthlessly dispatching question after question. Then I looked around some more and saw all these kids half my age beavering away, full of purpose, because all you Americans, apparently except as Gospel, this idea that the smart person is not the person who gets the right answer. The smart person is the person who gets the right answer the quickest. Mercifully, it ended at one thirty. Camille and I stumbled outside into the sunshine, clutching our plastic baggies. Our producer Jacob Smith was waiting. We had we had no ideas. Some people wanted to be lawyers. First of all, that was a shock. Um. Camille cheated because she she packed my We had to have these plastic bags. She packed my plastic back for me. But she gave herself like apples on like really really nutritious snacks, and I gots, which is the same thing. So you had fresh fruit and Malcolm, Malcolm had trail mix from Stevs. But but but who sharpened all your pencils, Malcolm and packed you a little tissues in case your nose runs. Okay, now, after both taking it, who like, what's your honest? Depending on who do you think did better? Oh? I think it Camille did. I was I did so poorly. I was fine, and I got really cocky and then I hit the logic games and basically I think I got zero right. I had no idea what to do. I sat there and I was like in a state of complete panic. I have I was untested until this moment, and now I have been tested. America has taken its measure of me, and I'm it's pretty it's pretty humbling experience. Why do Americans do this to themselves? Do they play scrabble with a stopwatch? In literature class? Do they get extra points for reading Tolstoy's War and Peace Overnight? Is there an oscar that goes out every year to the movie that got shot the quickest? I really don't get it. Okay, time to meander and digress. I became a Grandmaster of fifteen, So I think I became an im right after I turned thirteen, and I was a grandmaster of fifteen. So yeah, yeah, that was very, very good at the time. That's Hikaru Nakamura grew up in Westchester County, outside New York City. He's in his early thirties, although he looks about half that age. He's one of the best chess players in the world. How much chess were you playing as a kid? Pretty much all the time, so I was playing at the marshal at least I would say two or three days every single week, and then I would also be playing blitz on the internet chess club. I would say at least five to six hours every single day. Years ago, I met Magnus Carlson, who is the greatest chess player in the world, and I asked him how much he used a chessboard versus how much he just worked through chess positions in his mind, and he said, oh, I practiced mostly in my mind. In fact, I'm working on an opening right now. Hikaru Nakamura is a little like that. He's incredibly gracious and humble, but even when he's giving you what seems like his full attention, you get the feeling that there's a whole separate part of his brain breaking down. A be Fisher match from the early nineteen seventies. The first thing I did after my disaster with the l set was call up Hikaru, but I didn't want to talk to him about chess. I wanted to talk to him about time. So my first question about this would be what would happen within reason if there was no clock? How does the way that you approach a game of chess change if I remove the time constraint entirely? All right, So if you remove time entirely from the game of chess, every game of chess would be drawn because without without time, if you have an anilest amount of time to think about any given move. If I could think for thirty minutes on every single move, I do not think I would ever lose a game of Chuss to a human. To a computer, I would still lose, but to human even Magnus. Yeah, if I had an half hour forever move, I don't think I would ever lose to Magnus. This is why chess games have a time limit. Otherwise it's not a game. Tournaments would go on for months and everyone would end up tied for first. It would be like Little League. Everyone would get a participation trophy. So there's classical chess which is the kind played to the World Championships. Classical allots ninety minutes for the first forty moves, then thirty minutes for the rest of the game. Winning at classical chess involves calculation, working through many possible scenarios before deciding on a move. Then there's blitz chess. That's what Hikari was playing for five or six hours a day growing up. In Blitz, each player gets five minutes for the whole game. When you play blitz chats, it very much becomes about finding moves that look good, that are not blunders, that you can play almost instantly, where you use a couple of seconds in classical, how many moves would you go deep It normally would be about five to six moves and about three or four branches. It's a fantastically complex mental exercise, Yes, very much. I mean it's probably at least I would say, close to one hundred different different permutations of moves or a sequence. Is that you're looking at for every single move? Yeah? Yeah. In blitz we've truncated that process. Everything about Blitz and classical is the same same pieces, same board, same players, same choreographed openings, but the time limits are different, and what happens when you tinker with the time limit you get a completely different set of results. A classical chess Magnus Carlson is number one, he's also number one at blitz because he's a genius. Hikaru right now is eleventh in the classical rankings. But it blitz he's number two in the world. Why because he's really really good at the rapid pattern recognition that's necessary for blitz, and he's not quite as good at the complex calculation that's necessary for classical. Now, who's an example a good example of the opposite. Fabiana Fabiana Karwana is Actually he's probably the best example. I can't even think of anyone who is that much better at classical chest than they are at blitz and wrap. Why do you think he's not good at blitz and wrap? It um? I think? I think with Fabiano's it's the other way. He's very very good at calculations, so when he gets positions, he's very good at calculating and understanding what the possibilities are with more time, Whereas when he doesn't have time to calculate the long sequences all the way through, his intuition has to take over, and his intuition and natural feel are not as good as as everyone else. Fabiano is a tortoise, slow and steady. Hikaru is a hair ears back speedy. You construct your chest that favors the tortoise, or you construct your chest so that it favors hairs. I think you can see where I'm going with this. If we had a blitz tournament for the World Championship, you know, in two months, where would you put your odds of winning? I would put my odds of winning probably around twenty percent. I would say I put Magnus at about sixty percent. I put myself at twenty percent, So pretty good odds. Yes, so your your world changes if, just arbitrarily, we decided that the standard for international tournament chess ought to be blitz. Yes, it would change. Yeah, you would make a lot more money probably. Yes. Wait, there's a third variation, bullet chess. In bullet you get one minute for the whole game, and Hikaru is the king of bullet. Are you better than Magnus at bullet at bullet? Yes, a bullet yes, why now, so what's why wouldn't Magnus if he's if he's so serenely superior at all other kinds of chess. Why can't he beat you a bullet? In terms of the calculation what he does is like the sort Well it's not an algorithm, but sort of the way that he find tunes or the way he thinks about I think it takes a little bit more time to come to the conclusions. What strikes me is that the chess hierarchy formal chess hierarchy is an arbitrary function of the amount of time we have decided to spend on a chess match. Right, Yes, Like I said, I think you can see where I'm going with this. Hikaru is a hair, not a tortoise. And what is the l SAT. It is a test that rewards hairs over tortoises, which means that if the l SAT ran the chess world, they would consider Hikaru the greatest chess player in the world. They would crown him champion over Magnus el sat logic is that the best player is the one who solves the hardest chess puzzles quickest, and that's Hikaru. But that's insane. Not even Hikaru thinks he's better a chess than Magnus. What Hikaru would say is that he's a different kind of chess player than Magnus. And I haven't even mentioned puzzle Rush. It's an online game where you get a seri of endgame chess positions and you have to get to checkmate as often as possible in five minutes. Puzzle Rush is insanely popular. Do you play a lot of puzzle Rush? I have the highest score edits Yes, I do. Yeah, So what's your score? My highest score is fifty five, and what's the second highest? I think there are two people with fifty four, and then their couple with fifty three, and it falls off from there. Yeah. Yeah. We have most of the top players in the world played puzzle Rush. Most of them have. Yes, as Magnus played not officially, not officially, I think you might be playing and hiding it. He probably has. Yeah, yeah, yeah. Do you get any award for being the number one on puzzle Rush? No? They don't like crown u King of puzzle Rush. There's a three hour video of a cargo crushing puzzle Rush on YouTube. It is one hundred and thirty thousand views. This actually might be the most hotly contested form of chess you've ever played. You've never played something where millions of people are in the same tournament. Yeah that's yeah, Yeah, that's true. You're right, you're right. You could make an argument this is your greatest accomplishment. I guess, but I one't think of it that way. But yes, you're right, right, Yes. The only reason we don't consider that your greatest accomplishment is we've arbitrully decided only to honor chess played under these archaic rules in a tournament atmosphere. That also, rush is a I mean, if we decided to say that's what chess, yes, right, yeah, if yes, that's that's true. Yeah. It has many advantages. Chiefly is it allows the entire world to compete on an equal right. Right, Yeah, so you went against the entire world and you won. I don't know, yeah, yeah, I've never thought about that, but yes, that that is true. Yeah, but if one thing comes out of this, it is I hope you you give yourself a pat in the back. The order in which people finish in any cognitive task is an arbitrary function of how much time is given to complete that task. You can make it fast, or you can make it slow. The chess world has chosen to reward the tortoise. The L set has chosen to reward the hair. They've decided to play puzzle rush and reward the h Carus of the world, and not have us play classical and reward Fabiano. What does a legal world have against Fabiano? As I mentioned earlier, there's going to be a second part to this examination of the L set. Of course, there is why would I rush? In the next episode, I'm going to visit with the folks who administer the L set, the Law School Admissions Council of Newtown, Pennsylvania. But that can wait. Let's go back to William Henderson, who started me wondering about time and tests. The great justification for the L set is your score is supposed to be a useful indication of how well you will do in your first year of law school. It's a predictor. But Henderson's great question was what if the EL set only predicts law school grades because law schools make the same mistake that the L set does. In most law schools, grades are based in large part on how well students do on exams where they are deliberately not given enough time. You take cass Sunstein's Elements exam where you engage with the ideas of a legal genius and you have two hours. That's it. Henderson wondered if a law school doesn't rely so heavily on its students doing things quickly, if the school relies instead on take home exams and essays, what happens to the usefulness of the L set as a predictor. And he found that its usefulness declines. I'm quoting the data show that the L set was a relatively robust predictor of in class exams and a relatively weak predictor of takecome exams and papers. In other words, once you stop racing against the clock, then the people who do well on the L set no longer are the best at law school, which is exactly what every chess player in the world would have told you would happen. So what did William Henderson do when he became a law professor at Indiana University. He changed the way he evaluated his students. He started placing more emphasis on takecome exams, And when he gives an examined class, he makes it four hours, not two. That's completely open book, and I give out one of the questions in advance. I say, here's three questions. I'm going to test you on one, so you've seen one of the questions already. And then four hours. There's plenty of time to do the issue spotter, and I give a word limit. And I did that as a direct result of that LSAT study that I did, because I was cutely aware that you change the ordinal ranking when you pick a testament. And I say, you know, I don't think anybody will need all four hours, but if you if you want to take four hours, you're free to take four hours. Not long after he started grading that way, a student came to see him. This kid got an A on the exam, and so he comes in. He wants to talk about exams. I said, I've been here for three years. I've never done this well out of the exam before, and I want to know why I did this well. And I pulled out as an exam and it was an eight hour take home and I go, look at your first paragraph. You hit every single issue here in the first paragraph. It's just a well organized pearl. And the light bulb went on for him. It's just like if I had had more time all my exams would have been disorganized. You have a student who the system declared was of average ability. The student believed it, why wouldn't he. But then someone came along, someone by the way, with the great benefit of being a firefighter from the suburbs of Cleveland, who had the freedom to think a little differently, and he said, wait, maybe you aren't of average ability. Maybe you only think your average because we have chosen an arbitrary system to evaluate your ability that makes you look average. You are Fabiano and we had been making you play puzzle Rush. Oh I know what you're thinking. Yeah, yeah, yeah, but who won between you and Camille who got the higher score in the el sat? No stop racing ahead, you are engaged in uncomfortable listening. You'll have to wait until the next episode. What I want you to think about is that student of William Henderson's and his miraculous revelation that he really was a good student. After all, there are a ton of tortoises like him out there, not just taking the el SAT and sitting in law school classrooms, but competing for places at any number of schools and professions that have decided to tell their applicants whether they are or are not any good. And what I don't understand is how the hair has got to set the rules. I thought the whole point of the story of the tortoise and the hair was that the tortoise one. Clearly the kind of person who was most disadvantaged by this system is the tortoise. I realized, after my experience with the noodle guys and my time with Hikaru, that I'm on the side of the tortoise. I feel for the tortoise. I might be a tortoise. I come from tortoises, and we've all met tortoises in our lives. Who will not My mother is a little bit of a tortoise. She will not be rushed under any circumstances. She will not She does not make mistakes. She goes over things five times to make sure that they're perfect. She is ideal for a certain kind of work. She'd have a problem with the il set. She would say, why are you rushing me? And she would you know, and she wouldn't finish, and she wouldn't never guess. She can't guess. She's incapable of guessing. So my mom could never The profession is putting up a barrier to my you put a stop watch on thinking whatever, you rigged the system. So Camille wins fine, but when you go after the Gladwells, then it's personal. Revisionist History is produced by Mia Lobell and Jacob Smith with Camille Baptista. Our editor is Julia Barton. Flawn Williams is our engineer. Fact checking by Beth Johnson. Original music by Luis Guerra. Special thanks to Carl Migliore, Heather Fane, Maggie Taylor, Maya Kanig, and Jacob Iceberg. Revisionist History is brought to you by Pushkin Industries. I'm Malcolm Gladow. But Camille, Camille is like a smarty pants. She's going to do don't you think she's gonna do amazingly? Well? Yes, yes, Well, Camille has a natural um. She naturally prepares and is naturally persistent and determined. So I think those things will really help her a lot. Should I should I be worried? I think you should be worried. I think you should be worried. Yep

Revisionist History

Revisionist History is Malcolm Gladwell's journey through the overlooked and the misunderstood. Ever 
Social links
Follow podcast
Recent clips
Browse 166 clip(s)