What is the character in the film looking at? How does the object of their vision color how we interpret their countenance? In this episode of Stuff to Blow Your Mind, Robert and Joe discuss the Kuleshov effect from film theory and what studies in experimental psychology and neuroscience reveal. (01/27/2022)
Hey, are you welcome to Stuff to Blow your Mind? My name is Robert Lamb and I'm Joe McCormick, and it's Saturday. Let's head into the vault this week. It is part two of our series on the Coolest Jov Effect. This episode originally aired January. I hope you enjoy Welcome to Stuff to Blow Your Mind production of My Heart Radio. Hey, welcome to Stuff to Blow your Mind. My name is Robert Lamb and I'm Joe McCormick, and we're back with part two of our series on the Coolest Jov Effect. Now, as I explained last time, this is one that originally was going to be one episode. We ended up splitting it into so we're doing a little time traveling right now. This is an out of sequence introduction, but I guess from here we'll just jump right back into the middle of our conversation from last time. Let's do it well. Anyway, So I wanted to talk about a very interesting paper that analyzed the the history and meaning of the cool Shov effect and then also tried to recreate the Mojukan experiment. So this paper was published in the Cinema Journal by Stephen Prince and Wayne E. Hensley called the Coolershov Effect Recreating the classic experiment your nineteen. I think both of the authors on this paper were at the time professors at Virginia Tech. Stephen Prince is a is a film scholar who I know has done a lot of work on a Kirakurasawa. And I'm not going to cover the entire paper, but i just want to note some parts of it that struck me as as relevant and interesting. So they start off by telling the story of the Kolashov effect experiment, the experiment with that actor mojuk In making the neutral face and then either being intercut with with soup or with the woman in the coffin, and the audience is raving about how how expressive and powerful the emotions in the performance were. Now, one thing they do at the beginning is they note some differences in the details of the story that arise from different recountings of it, and so they end up casting doubt on whether the accounts of this experiment are first of all, historically accurate and second analytically valid, and so the author's right quote. The goal here is to provide a clearer contextualization of Kolashev's work distinguishing between its incontrovertible importance for an understanding of how cinema communicates and certain of its limitations, especially it's incautious merging of theoretical claim and observational assertion. As we will see, Kulashev may have been right, but perhaps for the wrong reasons. So the top line of this paper is that they try to recreate the Majukan experiment as it is usually described, and they do not produce the same result. But this doesn't necessarily mean that the broader implications of the Kolashov effect are wrong theoretically, uh, but it might mean something about the specific claims about a neutral face. Um. So they start off talking Aboutkulashev's belief in the power of montage and his arguments that editing is far more important the meaning generated by a film than the contents of the shots. So they talked about the masou Can experiment and then the other things we mentioned, creative geography and creative anatomy, and they described the general takeaway from the masuk And experiment as follows. Quote. Naturalistic emotive performances by actors were not considered by Kulashev to be essential to cinema because of the demands of montage, actors were to provide minimal restraint and fairly unambiguous gestural and facial expressions. As kola Chev puts it, quote, the presence of montage necessitated that the shots should be constructed simply, clearly, distinctly. Otherwise the flickering of a rapid montage would not be sufficient for a full scrutin any of its contents. And then the authors go on reacting, partly against the over emoting found in some silent films. Kulashov noted that quote, a preoccupation with psychologism rooted in the actor's performance was quite useless for the cinema. So in a in a lot of ways, it sounds like Kolashov kind of wanted to take the acting out of acting. He was like, there's too much psychology and acting. What we need instead is just sort of like shots of actors doing kind of like plain, unambiguous moments that can then be selected by the editor to insert in a sequence to make meaning of m Yeah, that's I mean, it reminds me of so many other discussions we've had about performance and direction. Uh. I'm always reminded of that that final sequence from A Geary The Wrath of God, where you have you have what ends up being a rather balanced and and and interesting performance by klaus Kinsky. But apparently it's because Vernon Herzag just wore him out, made him do take after take until he wasn't doing like a frenzied um uh you know, over almost you know, overacting overly intense performance. He's not raging, he's just actually you know, you know, emoting it at the level that the director wants and then can therefore be uh be used effectively in the edit. Yeah, and though if that story is true, it may have worked in this case. So I want to say I do not necessarily endorse directing by exhaustion. No. Now that was a special relationship obviously, But you often see this brought up, and you know, there's this idea of like, is this is it the is is this about the actor and the acting performance? Is it about uh editing? Is it about the director's vision? And you do often see that sort of push and pull be a you know, Klauskinski in vern Or Herzog or Jimmy Stewart and Alfred Hitchcock. Uh, you know, the the actor has a certain vision about how things want need to be, and then the director had maybe has another eye dea not not only about like this particular character in this particular performance, but how it fits into the the overall film, how it fits into the final edit. And so you could you can imagine somebody going into it with this this sort of very kulashav idea of just shoot, all we needed as neutral actors, we don't really need any of this emotion one way or another. And um, I don't know, there's probably some examples of filmmakers who tend to lean in that direction with very neutral performances. Yeah, you could almost look at that approach as uh, something that might be more common saying like music videos and stuff than in narrative films. Being probably find at some in narrative films as well, where the filming part of the filmmaking process is just sort of like creating a bunch of building blocks that can later be used in various arrangements to do whatever the director editor later decides to do with them. Yeah, it also reminds me how, you know, a lot of the films we're watching weird how cinema will sometimes feature non actors or you know, very very green actors, But the right sort of non actor can really excel in a scene if utilized correctly, you know, like not the kind of non actor where they're just really outrageous with you know, but but where they're just sort of very they're very neutral, they're they're almost barely there at all, and if enough of the other stuff is in the right place, it can really work. Now. I gotta say, though, as this paper ends up describing kola Chev's theory of film and montage, I think I can't agree with with what it sounds like Coolshov's vision actually was, because cool a Show apparently said things like the film shot is not a still photograph. The shot is a sign a letter for montage. So I think he's saying like a still photograph can have meaning on its own, but a shot in a movie is more like a letter in a sentence. Something which does not have meaning on its own, but is combined in sequence to make meaning clearly has some truth to it, because, as we've said, editing does constitute a major part of the the sense making or meaning making of a film. But I think that's also pretty overstated. You know, a lot of meaning lies in the editing, but the contents of the shots also stand alone to a greater extent and and matter a lot more than Kolashaw was giving credit here. Um. Though, again, to be fair, I think it's important for us not to forget that in the nineteen teens and early nineteen twenties, you know, film was still fairly young, Editing was still fairly new in cinema, and its powers were still being discovered. Uh. You know, it's like, like we talked about the very earliest films from the eighteen nineties and such, we're usually not edited at all. They'd just be one continuous shot. And even after editing was introduced, films of the Silent era typically did not have as many cuts as movies were used to today. Furthermore, the authors of of this paper argue that a theory comparing film to language is actually not super useful because there's just a lot of ways in which that doesn't work. Like film does things language cannot do. So you don't have to learn a language to appreciate the meanings of films. You you learn some conventions, but you know, you can just watch a movie and make some sense of it, even if you're not familiar with conventions. You to understand the language, you have to learn the language. Um. Meanwhile, language does things that film can't do, like photographic images used in a film cannot be recombined freely to make endless meaning the way a language can. There's also an interesting digression in this paper about Kolashaw being influenced by the ideology of industrial efficiency on the model of the American engineer Frederick Taylor, who was a big proponent of finding ways to make you know, production processes and factories more efficient, finding all the places where waste and and and problems creep in and eliminating those, And that Taylor's ideas of industrial efficiency were apparently very popular in the Soviet Union at the time, and that in a way, the authors say that you could view Kolashaw's emphasis on economy and acting as a type of industrial efficiency technique applied to film theory. Yeah, And based on what I was reading it, it doesn't. It does seem like a lot of his work was based in let's figure out what's working, and then how we can we can do that? How how do we make how do what is the most economic means of making effective film now? Ultimately, Prince and Hensley make the case that Koulishev really was trying to dress up his theoretical convictions about how film works with the imper mater of empirical science with this alleged experiment them as you can experiment, uh, And I think I'm pretty convinced by their description of it that way. I think this is something you've always got to be cautious of because obviously, you know, I don't object in principle to exploring or building upon artistic theories with empirical methods. But I would also say, my personal opinion is that a lot of these efforts to inject scientific methods into esthetics and and art and stuff can be confusing and unnecessary. Like I don't think you have to have an empirical scientific justification for an opinion about where meaning comes from in art or in film. Obviously, I'm a huge believer in empirical science. I just don't think it has to pervade every domain, Like aesthetics and art don't necessarily need scientific evidence and theories behind them that those fields just you know, work by different standards. And I think also a lot of times if you try to generate empirical scientific justifications for your beliefs about art or aesthetics or whatever, you're often just gonna end up doing sloppy experiments or drawing unjustified conclusions, even if you do a good one. Yeah. Um, like I'm reminded, you know of the fact that obviously you have a such thing. There's such a thing as outsider art and oursider cinema. Um, and and examples of outsider art and outsider cinema can be amazing, uh, you know. And on the other side of things, you don't hear as much about maybe outsider architecture, outsider structural engineering, things of this nature. Outsider medicine is probably you know, best avoided if you can, no matter how it's being dressed up. Well, I mean, I think empirical methods are good for fields in which you are trying to achieve very clearly specified goals, certain kinds of outcomes and get them as reliably as possible. And empirical methods are are less important in fields where you're you're just trying to be expressive or be creative and see what kind of emergent results come out. But if it's like like this turns my mind to like a b testing and focus groups used in film and television. Um. You know, not not necessarily a bad idea at all, especially when you're dealing again with a very mainstream product you want to appeal to a you know, a wide population of individuals. Um. But you know, there are plenty of arguments to be made about it as a potential, you know, sloppy experiment. As you say, perhaps one of the best critiques of all of this is that that episode of The Simpsons, the Itchy and Scratchy and Pucci Show, one of my favorites. It's just an old, creaky mirror. Sometimes it sounds like it's coughing or talking softly. Yes, But anyway, to come back to uh, Prince and Henley's description of methodological problems with the common descriptions of Kolashov's alleged experiment the Masukan experiment with the neutral face and the soup and the and the coffin and stuff. And they list a bunch of questions, they say, quote for such a seminal and basically uncontested study, there is virtually no information available about Kolashov's actual method and procedure. Did he, for example, will interview the subjects individually or in a group. What did he tell them beforehand about the purpose of the presentation, What, if anything, did he tell them about the nature of film editing or montage. What was the frequency of outlier opinions e g. People who did not think Masukan was saddened by the dead woman. Published accounts suggest the responses were uniform. Was this so? Unfortunately we do not know the answers to any of these questions. So, given these limitations, they attempt to recreate and try to replicate as best they can the conditions of the original experiment to see if they get the same result. So what they did was they put together a videotape that had some auditions for actors to produce a close up shot of a face that was just totally neutral and expression list And they had to go through a couple of rounds because in the first round the actor's neutral face was not perceived as neutral enough by the control group. Um. But so so they got a neutral face on a video and they did the same thing. They intercut it with a woman lying in a coffin, a girl playing with a teddy bear and a bowl of soup on a table, and they tried as best they could to follow Kolashov's cues about what what the cinematography techniques for making this work the best would be, so it would be UH people visible on a darkened black velvet background. Apparently the actors were told that they just needed someone to uh to model for an instructional video in which they would be required to do an expressionless or neutral face. So one difference is that instead of one long sequence intercutting with all of them, they did separate sequences for each reaction. So for example, it might go face soup, face fade out or face coffin, face fade out, and each shot was seven seconds long. And the separate sequences make sense to me because you might get a different reaction with some pairings than you would with others. So viewers each saw one sequence selected at random, and they were told that the experimenters needed help evaluating an acting performance. And then the viewers were supposed to select from a list of emotions that they thought were being portrayed by the actor. Options included happiness, sadness, anger, fear, Surprise, discussed hunger, no emotion, and other Apparently the participants were undergrads at a mid Atlantic university. I'm going to assume based on the author's affiliations, this was probably Virginia Tech. They said that interestingly, film students were excluded from the experiments since they might detect the connection to Kolashov and understand what the experiment was getting at, which could bias results. And in support of this decision, I mean, it seems like a good choice either way. But to justify this decision, they wrote about another recent attempt to replicate the Monsieur can experiment in France among film students who allegedly gave answers like the following quote. We know that the man does not change his expression, but because of the montage, we think we see him change or quote. We know the Cooleshov effect and it works. And then Princeton Hensley also had a control condition where they showed the face to twenty four film students, this time but without any inner cutting. They were just showing them the face by itself and asking them what emotion it was showing for the face that they actually used in the experiment. Percent said there was no emotion on the face. So this is a very good neutral face. You know that. That reminds me though of of use of neutral face uh sort of not still pictures, but just seen sequences where um, a character and individual is staring directly into the camera. Um. I'm thinking it's certainly about Ron fricksm Baraka, which features a number of these uh sequences where you'll you'll just have an individual from from one culture or another just staring into the camera. Or Another example that comes to mind is the film The Mission, where at the very end of the film there's you who just have several beats of one of the primary characters, uh staring into the camera and very neutral expression. And of course you have the entire film you've just watched to help uh inform your idea of what's going through that that character's head. Um. But but but still, it's it's a it's a great use of neutral expression, Like he doesn't it doesn't look particularly sad in that case, but you in you can see sadness in the character. You know. Well, yeah, that's a good example, but I think it also does raise questions about something that's supposed to be sort of outside the the standard interpretation of this of this experiment, which is like, well, wait, what are the actual contents of the face? Maybe that does matter. That's going to come up in the author's interpretation of the results they get. But so in the actual experiment they did, they had a hundred and thirty seven participants, including the control group. In the experimental group. In every condition, whether it was soup, coffin, or child, the majority of people said there was no emotion. So they saw the face that was supposedly neutral, they saw it intercut with whatever it was, the soup or the coffin, and they said, nope, there is no emotion on this face. In the soup condition, sixty eight percent selected no emotion. In both the child and the coffin condition, sixty one percent said no emotion, and so comparing that to the control group, in the control eight percent said there was no emotion, and that dropped down to sixty eight in the soup and sixty one in the child and the coffin. So you could say this is a small increase in perceived emotion, though the authors note that for the size of the group they tested, it actually doesn't reach statistical significance, so it might just be a random fluke. Furthermore, in the cases where the viewers picked in emotion, it was usually not the expected emotion, so it was not happiness for the child and so forth. So either way, this experiment find something somewhere between no effect and small effect on perceived emotion, which is a very far cry either way from Kolashev's reports about the audience is unanimous raving about the actor's subtle emotional performances, And so the authors say here that you know, in less contrary evidence emerges, it seems true to say that quote the Kolashov effect as reported no longer exists, even if the effect did play a role at one time, though emphasis there should be on as reported, because some of the broader implications of it probably do still hold true. Now, this raises an interesting question. If we assume, for the sake of argument, that Kolashev was basically reporting the results of his experiment accurately or with only slight exaggeration, what could account between the difference. Why did Kolashov get people raving about the subtle emotion in the neutral face, but that that didn't really happen in a modern experiment? The authors offer some ideas here, and I think they're all pretty possible via and certainly interesting. So one would be changes in audience expectation. You know, audiences today are accustomed to highly effective editing techniques that have been perfected over time, such as, like I mentioned earlier, the preservation of eyelines to enforce continuity of of perspective and reverse shots. Yeah, yeah, I think this is this is a big one. And I mean it comes down to like some of the basics of what we said earlier about how at least for many of us and certainly for me, like trying to watch an actual cooler show film is very difficult. Like it's just film has come has evolved so much since then, um, and and the effects are subtle in a way that you really the film only has to be even halfway competent to really just draw you in and create the illusion. Right, So uh so the author's right quote. It may be that a modern audience, by virtue of increased media exposure relative to cool A. Shov's day, has become accustomed to a more systematic and complex set of associate creational cues, such as those supplied by the continuity system of editing. And is correspondingly less likely to respond to a montage sequence that employs a blank face and minimal, if any associated cues within shots. So maybe the bar for perceiving emotion in films has has gone up, you know, it's just harder to do now. And at the time the Kolashov did his experiment, allegedly maybe the audiences were just we're just more it was easier for them to project that emotion now that There could be a number of ways to read that. One way is is thinking about how much exposure modern audiences have to modern editing techniques. Um the other way, I guess, and the authors don't really favor this explanation, but they say another way of looking at it is naivete on the part of the early audiences. There's some kind of projection going on, because maybe early film audiences were just so bewildered by moving pictures that they almost like hallucinated projections of emotion. And uh, the authors don't think this is a very good explanation for one thing, because they argue that a lot of the stories that are used to to illustrate the sort of bewilderment of early film audiences, like that, you know, the semi mythological things about the audiences running away from the Loomi air train and stuff that they say that I mean, there were sort of events of this kind, but they have been mythologized in a way that over emphasizes how naive early audiences were, and that a lot of these kinds of reactions may have just been audiences playing along there at the theater having a good time, and they're playing along with what the suggested reaction was supposed to be. That's true once you especially when you're dealing with a group of people, you know, watching watching anything with a group, even even today with our our modern exposure to cinema, you know, if one person jumps, everybody can jump. That sort of thing, you know, you're more maybe you're more likely to to laugh or scream if you're watching it with with other people. That sort of thing makes me think about William Castle and the Tingle are trying to get people screaming in the movie theaters. Yea, yeah, which which is uh is infectious. As I think I mentioned in that Tingler episode, I got to see the Tingler UH in a theater and people were totally playing into it, like it's still worked today, so good. Okay. A couple of other possible explanations for the difference between Kolashov's report and then and then they failed attempt to replicate those findings. Another one is response bias. So this seems quite possible to me. Maybe it was originally a sloppy experiment. Maybe Kolashov primed his test subjects to react the way they did and they complied. Uh. You know that This is why double blind tests are very useful. If the person administering the test doesn't know what hypothesis is being tested, it's harder for them to behave in a way that would bias, that would bias the subject response in favor of it. And there is of course extensive evidence that Kolashov was already committed to his theory about the power of montage, but for he allegedly conducted this experiment like he he already had the result he was looking for in mind. Yeah, like the neutral face. I keep thinking of examples now of neutral face or very neutral or or just you know, low key acting performances, And one that instantly comes to mind is the sequence in The Godfather where al Pacino's character is in the restaurant with uh, what is the the corrupt police officer and Sterling Hayden and yeah, and the turk, right is that the other character his name also it's also um And of course what's gonna happen is he's gonna go to the toilet, he's gonna come back with a gun, and then he's going to shoot them both. That's the plan. And there's that great sequence where you see al Pacino's face and he's he had a very again, very neutral expression, and I previously just always thought, well, that's just he's just he was such a great actor at that point in his career, Like like you can just see the wheels turning, you can see all the tension going on behind the scenes. But maybe not, maybe he's just thinking about, you know what, what what he needs to pick up at the grocery store later on in the day, and it's just all about everything else going on in the scene and how it's been put together that could be there. They're actually a number of shots in The Godfather in particular where they're memorable because of al Pacino's expressionless face, like when when Carlo Ritzie confesses at the end to having killed Sonny, and Michael just looks at him with the blank expression. But you read a lot into that blank expression. It is a murderous blank expression. But there's another way of reading the al Pacino example here, and also of possibly interpreting the original Mojoukan experiment. I really like this explanation. What if Kulashov's montage was loaded with more conventional emotional content than he claimed. There could be a million ways this could be true. But for example, what if there was something special about the face of Majukin. What if there was something special about the face that Kulashov used in this supposedly neutral test film, there was less neutral than we would be led to believe. The authors of this ninety two paper note quote there is a difference between an expressionless face and an ambiguous expression. And they started an experience from their own experiment. They talked about how the very first tape they created, if somebody trying to do a neutral face had to be rejected and replaced with a different actor because it failed to be rated as neutral in the control condition. So that was the control when there were no shots juxtaposed, the control group thought they perceived a range of emotions in the first neutral face they looked at, and then the author's got a different tape, different actor, and it succeeded at being perceived as more neutral in the original control. This is great to point out, Yeah, the difference between a neutral face and an ambiguous face, because obviously this is one of the arguments for why the Mona Lisa by Leonardo da vinci Is is such a an admired piece of art. Is because you can easily read what the Mona Lisa is um is thinking or feeling, but that she has this ambiguous countenance, right, And the difference would be that there there is a difference between ambiguous and neutral. Neutral is something we look at and we see I I don't see any emotion on that face. Ambiguous is you see emotion, but it's not clear what it is. It maybe suggests something that could go in different directions. Oh, but then the author's come back to talking about this more ambiguous, more emotional face that they got the first time they tried to record a tape, they said quote. When other viewers were shown this face and sequence, many attributed a wide range of emotions to the actor, some consistent with the coolishev effect others not. The sequence with the soup, for example, elicited interpretations of apathy, disgust, contemplation, detachment, dislike, indifference, lack of interest, as well as an occasional attribution of hunger. The ambiguous expression see him to offer a stronger interpretive cue for the viewer than did the expressionless face. If cool a Chauvian montage may not be capable of making an expressionless face emotive, it may very well do with an ambiguous expression, since the objects like soup, coffin, or child provide a context for resolving the ambiguity. And I think this interpretation seems very likely to me because again, the allegation is that Mozukin was a famed actor, and so there's naturally you can imagine a famed actor's face has something special about it's kind of brimming with with the the implication of emotion, even when they're being relatively subtle or not, you know, offering a big smile or frown or whatever, right right that this may well have been the sort of performer that was highly aware of what their face is doing. That is, you know, that is practiced in front of the mirror that knows what they're projecting, and therefore to you know, to a certain extent, might be incapable of a neutral face at least when when when told to pull some sort of face. Right, So if there's something to this interpretation, I would say that that the coolest off effect, even in the specific case of interpreting neutral faces, as you know, based on the the editing context, it's absolutely tapping into something real, but there might be like thresholds or limits, like there is some truth to it, but it can't overcome some truly deeply blandly neutral faces, you know, like some ambiguous faces just offer more hooks on which to hang emotional values created by the context. Yeah. Yeah, I also wonder what would what would happen if you you took exceptional faces and you threw them in, you know, and not necessarily even exceptionally dashing faces, but like just exceptionally evocative faces, like like I don't know, like a Peter Laurie. You know, if you put Peter Laurie in there, just even you know, even though he's gonna do you know, a new traw ambiguous face. Uh, you know, what would happen to the experiment of course, in that case, you'd also have to not know it was Peter Lori, because then you're gonna you're gonna start typecasting, like, oh, we know what kind of guys this this this actor plays. Yeah, you'd be suspicious, you'd be reading negative emotional or suspicious mind content. What is the planning for that soup? He's going to poison that soup? Isn't he right? Anyway? I think the authors make the point in the end that the the broader implications of the cool A show of myth that that individual shots, which may be low on meaning or emotion by themselves, can become highly charged with meaning by the power of the surrounding editing. This is obviously true, and it is largely the basis for the magic of cinema, But the specific claim about supposedly neutral faces appears to be not true, at least for some audiences or some faces. But this raises really interesting questions like, what are the properties of the maximally cool A show ambiguous face? You know, what, what kind of skills would you want an actor to have to be able to have these you know, subtle ambiguous expressions that can be sort of driven any which way. By the surrounding context, by a bowl of soup or by a coffin. I guess, you know, I'm just guessing here, but at the bare minimum, you need to have some sort of like spark of attention, Like they're saying, it's not not enough perhaps to just rely solely on the editing to imply that there's a connection between this shot and the other. But the person's face appears to be looking with interest at something. You know, Yeah, that's that's a good point. I mean, I think sometimes with these studies, like the face doesn't just look neutral. It looks like it's not seeing anything, right, Like if it's just like mug shot and then and then pick a plate of spaghetti, Like okay, you show me a mug shot, and you show me some spaghetti. Maybe something that's crucial is that the even if they're not showing a very clear emotion, that it looks like they're looking at what ever is being shown. So Princeton Henley is very interesting, but it was by no means the last study on the cooler shop effect, the last attempt to look at it empirically, And actually since then some other studies have kind of come back on the other side have found a little more support for the original alleged finding. So one example is the is the study by Dean mobs at All from two thousand six called the Cooler Shop Effect the influence of contextual framing on emotional attributions. This was in social cognitive and effective neuroscience, and the test here was a little bit different, but they did basically look for the same type of effect and did succeed in producing it experimentally. So in this case, they didn't use just a single supposedly neutral face as the stimulus. They used neutral faces and then what they call faces displaying subtly fearful or happy facial expressions, which if you want to look up the study you can see the stimuli they use the yeah, they're they're play their faces that are almost neutral. They've just got the barest little hint of a smile or kind of an apprehensive frown. And then they put together a task where they would actually they paired it with neuroimaging in the study, so they'd have people doing neuroimaging while they gave them the task to look at this face and then imagine that the person is watching a movie of various kinds. It could be a happy movie scene or a scary movie scene. Uh. And they did find that people were, on average more likely to interpret neutral or only very subtle expressive faces more in alignment with the emotion that you would expect if they believed the person was watching either a scary or a happy movie. And so it's worth noting that there is an effect here, but it's not as shockingly powerful and unanimous as like those original tellings of the Cooler Show experiment would suggest. Mm hmm, yeah, this is interesting. This is something we'll continue to look at. I also like that they were looking at scary and happy movie scenes because it also brings to mind episodes we've done in the past on audience reactions too scary movies and how oftentimes, like like the the reaction you have to a pleasant movie or certainly a funny movie compared to that of a scary movie. Uh, that they may be more like than one might think. Oh yeah, because a lot of times people laugh when something is scary. Yeah, laughing, Uh, you know, reacting to the way that people around them are reacting. And if you are acting frightened during a frightening movie, it's I feel like, it's very often a kind of excited frightening you know, that's safe kind of like I am. I am afraid for the characters, but I'm not necessarily afraid for myself. You know, I've actually wondered before if so. A lot of my movie going entertainment pleasure comes from watching be horror movie is essentially as unintentional comedies and having a good time laughing laughing along with them. But I wonder if part of that grows out of a kind of defense mechanism learned in childhood, that that I could protect myself from something scary if I sort of forced myself to see it instead as something funny. Yeah, I don't know. I I certainly catch myself going like ah, more like that exact um sound if it is say a slightly goofy or goofy monster that is suddenly jumping out as opposed to a more I don't know, effective looking special effect. Uh there's something about I don't know, it's probably you know, all this is highly subjective, but for me at least, uh, you know, maybe I'm just leaning into the imagination more in those cases. Now, just very briefly, I wanted to point out a couple more studies I dug up that looked into the cooler Shov effect more recently than this one. So there was a study in the journal Perception in in two thousand and sixteen by Daniel Barrett at All called does the cool a Shov effect really exist? Revisiting a classic film experiment on facial expressions and emotional context. So they note some of the stuff we already did, doubts about the original experiment, and then the fact that recent attempts to reproduce the effect have had conflicting results. So they tried it out with a group of thirty six participants who were presented with twenty four film sequences of neutral faces across six different emotional conditions, so trying to reproduce the same effect, and they actually did find a correlation. It may it may not have been huge, but they said quote for each emotional condition, the participants tended to choose the appropriate the appropriate category more frequently than alternative options, while the answers to the valence and arousal questions also went in the expected direction. So they did find a mild existence of the cool A Shov effect in their research here and then there was Another one by Baranowski and Hate in UH Perception in two thousand seventeen, called the auditory cool Ashov Effect multisensory integration and movie editing. The study tried to see if there were any cool a Show of type effects, not for cross cutting with visual images, but for music. So the question is does music affect what emotions people detect on other people's supposedly neutral faces, And according to the authors of this study, their results were yes. They found that sad music did in fact make people more likely to rate a supposedly neutral face as sad and vice versa. Well that that that doesn't surprise me at all. I mean, music, especially music and film, is highly manipulative at times. And uh and I think we've all seen experiments with this sort of amateur experiments with this online where you take, um, Johnny cash Is cover of nine inch Nails Hurt, and you play it in the background of virtual virtually any uh, neutral or ambiguous footage, and you're going to get a sense of like deep personal anguish and and hurt. I'm just I'm just putting it all together in my mind right now. I'm seeing I'm seeing clips from like happy Gilmore or something, but with with the Johnny Cash Yeah, to see if I still feel. And then finally one last one there was a paper by mullinicks at All from twenty nineteen in pl Os one that also looked at the cool a Shov effect, trying to see if it existed for still photographs instead of dynamic film sequences. And the authors say, yes, they did the cool Ashov type experiment, but just with still photos, and they found there was in fact a cool Ashov type effect for just for still images. Okay, also not surprising to me anyway. So it looks like more of the recent studies into this have found some kind of effect, though I think sometimes the effects are, you know, the kinds of things you're more likely to normally see in psychology experiments, kind of modest effects rather other than the overwhelming unanimous effect described in the the original Masoukan experiment. Now, I'd like to take um all these points we've been hitting and come back around to something that I briefly discussed, and that was Leonardo da Vinci's famous sixteenth century painting The Mona Lisa. One of the most intriguing aspects of this painting is the the ultimate ambiguity of the expression. You know, the Mona Lisa smile especially, Uh, it's a it's a it's a it's a slight smile. It's a kind of an ambiguous smile. What is she smiling about or beginning to smile about? Um? You know there there there have been a number of papers written about this, and I'm certainly not going to do them all justice here, but I wanted to touch on some findings that I think can potentially contribute to this conversation. Now, wait, did this originally come up in our making a distinction between neutrality and ambiguity and so so that maybe you're suggesting that the Mona Lisa's face might be one of those famous face is that is ambiguous but not neutral? Right, it doesn't look like a death mask. But also you know, she's not she's not scowling, she doesn't look like Vigo the compathion. She's not smiling ear to ear. It's a very interesting expression, to say the least. Um that people have been discussing and studying for for for decades and for for ages. Uh So I'm not going to cover all the studies, but there there've been there's been plenty but I was looking at one. This was a theory that was put forth by Professor Margaret Livingstone of Harvard University UM. She argues that, UM, a lot of what fascinates us about this painting is because the smile appears differently depending on where you're standing in position to the painting. So if you look at it with your fobial or direct vision, uh, then arguably there's not really a smile going on there. But if you view it from your with your peripheral vision, out of the corner of your eye, then it seems like there's a pronounced smile. Now this doesn't This this little tidbit doesn't particularly I have a lot to reveal about the broader topic we're discussing here, but I found it interesting just talking about. And indeed, it's one that you can You can pull up an image of the Mona Lisa on your computer or your phone, or if you have a copy hanging in your your your house. You can do it this way as well, and you'll find I think that you do get this effect if you kind of look at out of the corner of your eye, it seems like there's a pronounced smile. Look at her directly and uh, it's it's not there. I see exactly what you mean. Another interesting thing is that my mental image of the Mona Lisa is smiling more than the actual image seems to be when I look at it. Yeah, something about the lower resolution copy in my brain appears to have accentuated the smile. And maybe somehow that's picking up on the kind of subtle shade ding of the contours of her cheeks which looks like they could be continuing the lines of her mouth, but it's not her mouth. Yeah, yeah, so yeah, I think that that's that's very much it. And of course you can get into deeper discussions of you know, to what extent um you know this is intended and you know what Leonardo Vinci's trying to do with this, um Because another another aspect of the smile that's frequently brought up is that it's um uh, it's it's not a symmetrical smile. Um. And this is often cited as is one of the key interesting aspects of the Mona Lisa's smile, of Mona Lisa's face in general, um. Now, the emotional impact of her expression has been much debated over the years, and he is like like a lot of what we discussed in part one and in this episode. It's one of those areas where you can you can science it all day, but you're still working with subjective art rather than objective principles. But there are some papers that I think have some revealing information based generally on you know, small studies looking at asking people to look at the painting or look at portions of the paintings sometimes they've been manipulated in a key way, and see what people have to say about it. And this is where we're getting, uh, you know, we're getting into something that's more in line with the broader topic here. When you look at the Mona Lisa, what kind of emotional um understanding is passing between the painting and yourself? Does it depend on what painting is across the room from her on the other wall, so like what you're perceiving her to be looking at. They didn't get into that, uh as much, but I couldn't help but think of it. I kept thinking of her looking at soup and so forth. But one paper I was looking at was a twenty nineteen paper from Marsilli at All published in Cortex, the journal Cortex in which the researchers asked forty two individuals to rate which of the six basic emotions as well as a neutral expression of emotion was related in chimerical images, uh, constructed from the photos. So chimerical images in this sense are formed from opposing halves of a pair of same or different faces, usually in like studies in courtroom settings. But in this case it would be like you know, um, my understanding here is like mirroring different parts of the face, stealing with the with the asymmetry. You know, like what if you had side A is the and you just cloned it onto side B. That sort of thing. Now, the results in this case indicated that happiness is expressed only on the left side of Mona Lisa's face, not on the right. Uh. And this actually leans into the interpretation that the Mona Lisa's smile is not a legitimate smile at all, but a fake smile uh, something that is either you know, a noteworthy subject of of the art in and of itself, or has a more specific, even cryptic purpose in da Vinci's art here, but and I think potentially makes it more interesting. Peace it's not of just a painting of a woman smiling. It's a painting of a woman pretending to smile, yeah, faintly. This is interesting because I know that's something I've read, and I don't know how legitimate this is, but I've I've at least read um facial expression ambiguity as one of the features people use to detect fakeery of emotions in others. So when people look at somebody else and they see that their smile is asymmetrical, they're more likely to think they're faking it, right right, um. And this is a topic we've we've covered on the show before because you get into that whole topic of of micro expressions and reading micro expressions and uh, the the idea that that a fake smile looks one way, but there's a more profound, pronounced um muscle definition to a legitimate smile. And so that's, I mean, that's on it on its own, is something we might take into account when considering ambiguous like semi happy, semi smiling, ambiguous uh images and ambiguous faces used in one of these experiments. Now, another study I looked at here was one from seventeen by leacci at All published in Scientific Reports. The researchers here manipulated This one's actually kind of funny. I think manipulated Mona Lisa's mouth curvature. Uh, and studied how a range of happier and sadder face variance influenced perceptions of her emotions. So, um, the actual paper gets into a lot of like they bust out some equations and math on this, but basically they're just doing what you're imagining now, like making the smile more pronounced or making it less pronounced. And um, they were able to manipulate perception along a sadness happiness um uh spectrum, but contended ultimately that their data indicates that the natural Mona Lisa, at any rate, is always happy. But I found this more telling quote observers recognize positive facial expressions faster than negative expressions. Uh. This is not a finding, but just a reality that they were discussing in in the paper. So in other words, faces spiraling down through neutrality, ambiguity and into other emotional states require more contemplation. Uh. And and I'm making assumptions here, but but more nuance. So like the like the face that's smiling ear to ear or is in a you know, the vego of the coppathion scowl. We don't have to think long and hard about that, like what kind of emotion is this person having about the soup. We know that they they're either ecstatic over the soup or they just hate the soup or something involved with the soup. We don't have to uh to think about it much. But when you have that that that ambiguous smile or even a slight uh frown, you know, that's that's when that's when that really makes you think, like what is this person thinking? My my theory of mind has to maybe engage more to try and figure it out, and then ultimately we have to remember, I mean, one of the key things about people's faces is that the face itself is a communication array. So like we're trying to get information potentially about that soup, right, like like that this individual might know of that soup is good. I want to know like what the inside track is on the soup um or on other human beings before I myself decide how I feel about it. I know this is sort of besides your main point, but it also makes me think about the strange biological contingency that one of the main features of that communication arrays also the whole that soup goes in. It's true. Do you ever think about how weird that is? You know, didn't have to be that way, but we just we we cram in, we cram in nutrition and speak through the same orifice. It's weird. It's true, it's weird, But you know, it's always a reminder that we shouldn't try and do both at the same time. But to bring it back to Koloshov, I do think this drives home a little bit of the susceptibility of ambiguous faces. You know that we can if the face is ambiguous, we have to think more about it, We have to think more about the context. But you know, what is the relationship between um shot A and shot B, right? I mean that would go along with what mobs that all said in their background again, which is that, you know, the broad finding of behavioral researches that people rely most on context to interpret the faces of others when the clarity of the facial expression is low, so that could be ambiguity or other things maybe or maybe just like it's hard to see, and when the clarity of the context is high, so when there's information in the context and less information in the face you reach for the context. Thank you well anyway. I guess this all brings us back to one of the questions posed by the Prince in Hensley paper, which is, you know, I wonder if certain actors are just more likely to um more likely to give rise to this effect than others are. And that again drawing on that observation that there's actually a different ins between a neutral face and an ambiguous face. I was trying to think of examples of actors who's what you might call blank or neutral faces might tend more toward expressive ambiguity rather than true neutrality. So even when their face is supposedly at rest, you could look at it and and it would seem valid to interpret a wide range of intense emotions to them. The best example I could think of, and I didn't pick him just because I love him as an actor, though I do. The best example I could think of was Toshiro Mufune, who you might know from a cure Kua Sawa movies. You know, he's the star of your Jimbo and movies like that. I would say he is somebody who, even when he's doing something very stoic with his face even when his face appears to be at rest, you could easily imagine that it is expressing a range of diametrically opposing emotions. And Rob I I pasted in a picture for you to look at here. That's just a portrait of him. I don't think this is even from a film. I think this might just be like a studio portrait still, because this is one where I've seen, you know, like that he's done autographs on and stuff. To my eye, in this portrait, he could be happy, he could be sad, he could be affectionate, he could be hungry, he could be angry. All seemed totally plausible with the expression on his face. And I guess this seems to correspond with the fact that I'd say he's an actor known simultaneously for having a highly emotionally expressive face and for often playing kind of stoic characters. Yeah. Yeah, you think about the especially some of the samurai type characters that he played, it attends to be an intense stoicism to those characters. Though at the same time, I mean, you think of his the McBeth character or the equivalent of McBeth pretty wise and Throne of Blood, you know, certainly he's you know, there's plenty of wide eyed crazy shots in that film, especially towards the end. But yeah, a lot of a lot of the characters he plays, how have a certain sternness, a certain stoic quality. Uh that that has ultimately has an intense ambiguity to it, And it makes me think about a difference that. You know, sometimes you read psychological studies that are measuring emotions in some context, and they measure emotions in terms of both valence and intensity, where valence means what the emotion is, so it could be like positive emotion or negative emotion, and intensity is how strongly it is felt. Thinking about this makes me wonder if maybe there are some people whose emotional expression naturally tends to be high in intensity, even when the valence is unknown or unclear, If that makes any sense. Yeah, yeah, so I wonder if that's especially the kind of person that you use a picture of, that kind of actor trying to do a neutral face. But then you do a Coolishov type experiment and people would be like, yes, you know, you show them looking at the coffin, they're very sad. You show them looking at the soup, they are ravenous, whereas there are other actors who whose face is just more successfully convey a blank neutrality where people see it and they say, I don't think this person is feeling anything. Yeah, yeah, I think it's a good point and to try and sort of prove it out for our own purposes. You posted this picture of a man in our notes, and I posted a picture of soup next to him. And indeed, if I look at the two and I sort of go back and forth, if yeah, I can read, I can lean into different interpretations like is he he is angry that the soup has been served, maybe it was served too early, or it's you know, it's clearly cold, or he just had the soup yesterday and therefore he has uh he is I rate? But he also could be like, yes, now it's time to to really get into this soup. Yeah, yeah, or or various other interpretations. You know. Weirdly, some of the other actors I know who fit into this mold are not just film actors. I mean a lot them are film actors, but especially people who have done like modeling, like fashion modeling or art modeling like Grace Jones comes to mind as somebody who could have have a facial expression that is ambiguous in valence but high in intensity. No, yeah, I definitely, yeah, I definitely can see that with Grace Jones. I was thinking, I was trying to think of good examples of this, and uh, like my mind turned to some actors who certainly, you know, have kind of like a smoldering uh stare or have you know, good at the stoic type characters are especially the sort of Joe cool characters, you know, as I think of them, where you know, it's like it's playing some cool, cool dude is like a detective or something, and he's you know, he's acting pretty much unfazed by everything around him. But I think the better example I ended up turning to is Harry Dean Stanton, who often played very you know, very sort of emotionally muted characters. I would say, though not Joe cool characters, you know, not not a character or that's so far above it all that he feels completely at ease. Oh, I think Harry Dean's potentially another great example. Yeah. Yeah. And another like actually kind of like a suite of answers that came to mind were from the uh, the the Alien film franchise. The various actors that you had playing androids. UM, specifically thinking of Ian Holme, Um, Lance Hendrickson, and Michael Fassbender, all three very talented actors, um but um, But in all cases they're supposed to be playing this artificial human type of being that has no emotions but but has an intent and in depending on which film you're landing on, in which particular incarnation of of the android, that intent maybe um benevolent or or might lean more neutral or might be malicious. UM. And Yeah, I don't know if i'd go there with Ian Holme actually, because Ian Holmes seems unusually capable of projecting absolute blank neutrality where you don't get that that ambiguity that spins off in all the directions. Like I think he would be he would be great to have people like absolutely fail to reproduce the coolish of results have him doing blank face. But other ones you're saying, I agree, Yeah, so I don't. I don't know. Like I was just thinking back on those films, and even though these are the characters that are not supposed to have emotional states, in some cases, I feel like I have a better handle on their emotional states versus other human characters in those pictures. Yeah, but I have to admit I did not paste all of their photos into our document and put them opposite soup, so I haven't tested it myself. Oh you did put fast spender next to soup, though, And I gotta say, he looks hungry. Yeah, yeah, yeah, he looks. He does look like he is, um, he's about to dine on some soup. Can't you just imagine a scene of him sensually teaching his twin how to peel above or nut squash. Yeah, that would be good, feeding each other's soup with wooden spoons. Yeah. Well, anyway, all this is just to say, and to be fair, maybe some studies have done this and I didn't realize it, but it seems like maybe one good move to try to avoid the the the the interactor effects of the of the stimulus you use in Kolashov type experiments is to just like get a whole lot of pictures of neutral faces and then serve them up at random, and so you can get kind of the neutral face photo averaged out over a big population, instead of having it fluctuate based on like how truly neutral your supposedly neutral face looks. I'd be delighted to hear from listeners out there what their thoughts are and their specific examples, uh, from cinema and from you know, the faces of various actors. You know, I wanted to come back to something that which which I thought is kind of interesting about this. Uh. Even if you'll accepted that the cool as show of effect is rather modest or only applies sometimes, it is still pretty interesting that it indicates how flexible the human brain is at constructing artificial scenarios and still applying like human logic to them. That like, you know, you're not observing a real scenario in life where you're trying to guess if somebody is hungry. You're looking at a photo or you're looking at an image on a on a screen, and then it's being intercut with a you know, a coffin that they might be sad at, or a just a picture of soup or something, and we we start applying the same logic we apply to real life to these obviously artificial stimuli. Yeah. Yeah, And I think it's a great reminder of just how film works and and and other mediums of virab but especially film, how you know there they still require a viewer. And if there's not a viewer, uh, there's not a movie go or there's no film experience, since therefore there's no film, and so there's no matter how polished the thing on the screen is, there's something that takes place not only between the film and the viewer, but inside the viewer's mind. That's that's critical, and that a lot of times we don't notice how many gaps were filling in as film viewers, Like, Yeah, you don't realize how much work you're doing, and it's work that is apparently pretty easy to do. It's just something we we tend to do pretty much automatically while we're watching movies is fill in those gaps of logic, make connections between one image and another, make assumptions about what's going on in an actor's head when they're portrayed on screen based on the context or the music, you know, what was shown just before after. But it's one of those things where it gets pretty weird when you start to notice all of those like assumptions you're having to make and mental work you're having to do for a movie to make sense, which in reality is a flickering succession of moving images, which you know, sometimes if you were to be very literal roll about them, are are totally unconnected. Like you see like a staircase that's from one state and then a house that's from another, and then somebody's coming in through a front door, and you just connect it all is this is all in the same place, persons just moving through their their daily routine. Yeah. We often think of of viewing films and watching TV programs as being kind of a shut your brain off kind of a situation, at least with certain types of of film and TV. And you know, we think that, Okay, if it's a it's a highly crafted product, we're not gonna have to mainstream product, we're not gonna have to do much thinking. It's gonna hold our hand the whole way. But but yeah, even even in the case if you're sort of you know, by the numbers summer Blockbuster, uh, you know, very much repeating a plot you've seen before, with the sort of characters you've seen before, your brain is still filling in these little gaps, like you say. But on the same hand, I think one one thing we can drive home based on what we've been discussing here is that that the opposite, uh, in a way is true, is that if you're dealing with a film that's say, is uh, you know of a of a genre you're not that familiar with, or a time period of filmmaking and out of familiar with. Um, perhaps it's a you know, more more of an art film, or it's you know, foreign language, etcetera. A lot of it is still going to come down to human or humanoid entities interacting with things in each other, and then our brain is going to make presumptions about their mental state and their emotional state. Oh yeah, yeah, you you infer drama even when the thing you're looking at is almost actively resisting it, and that that goes beyond movies. In fact, I mean what is drama. Drama is somebody wanting something or trying to get something and then coming up against resistance in some way. Uh. People infer those kinds of dramas on like balls rolling around on the table. They're literally studies of that. You know, people will say, like the ball wanted to go down in this hole, but it you know, it couldn't get there because something was preventing it. All Right, we're gonna go ahead and close it out there, but we would love to hear from everybody if you have particular thoughts on the clue Shov effect. Various examples and studies we've discussed in these episodes. Uh, some of the various examples from film and acting that we have alluded to. Perhaps you have some better examples that you would like to bring to our attention. Just write in and let us know. In the meantime, if you would like to check out other episodes of Stuff to Blow Your Mind, check it out in this Stuff to Blow your Mind podcast feed. You'll find that wherever you get your podcasts. We have core episodes on Tuesday and Thursday. We have a listener mail on Monday, short form artifact episode on Wednesday, and on Friday we do Weird How Cinema. That's our time to set aside most serious matters and just discuss a weird film. Um. If you want a quick way to get to our podcast, you can just go to Stuff to Blow your Mind dot com. That should still redirect you over to the I heart listing for our page. Huge thanks as always to our excellent audio producers Seth and Pollis Johnson. If you would like to get in touch with us with feedback on this episode or any other, to suggest a topic for the future, or just to say hello. You can email us at contact at stuff to Blow your Mind dot com. Stuff to Blow Your Mind is production of I Heart Radio. For more podcasts for my heart Radio, visit the i heart Radio app, Apple Podcasts, or wherever you listening to your favorite shows.