Emotion Recognition: Biases and Distortions
Based on materials from Halberstadt, J., Winkielman, P., Niedenthal, P. M., & Dalle, N. (2009). Emotional conception: How embodied emotion concepts guide perception and facial action. Psychological Science, 20, 1254-1261.
How do our biases shape the way we perceive reality and those around us? And why is recognizing emotions not our strong suit? Scientists have been exploring these questions, revealing just how much our preconceptions can distort our interpretation of others’ emotions.
How Our Expectations Shape Perception
Many remember the Soviet science documentary “Me and Others,” which demonstrated through experiments how suggestible and dependent on others’ opinions we can be. In one experiment, participants were shown a portrait of the same person, but were told beforehand that the person was either a scientist or a murderer. When asked to describe the person, those who thought they were looking at a “murderer” saw cruelty, a sly smile, and secrecy. Those told the person was a prominent scientist saw kindness, integrity, and intelligence in the same face.
But is this just about suggestibility and conformity? Recent research from the United States, New Zealand, and France confirms that our initial impressions of others’ emotions distort both our subsequent perception of their facial expressions and our memory of them. In other words, once we interpret an ambiguous or neutral expression as anger or joy, we later remember—or even actually see—that emotion in the face.
Do We See Reality or Our Own Biases?
As Professor Piotr Winkielman, a co-author of the study from the University of California, puts it, the research addresses the age-old question: “Do we see reality as it is, or is what we see shaped by our own biases?” The results show that what we think has a significant impact on what we perceive, making emotion recognition subjective and unreliable.
Another co-author, Jamin Halberstadt from the University of Otago in New Zealand, explains: “We imagine that our emotional expressions are an unambiguous way to communicate feelings. But in real social interactions, facial expressions are a blend of several emotions and open to interpretation. This means two people can have different memories of the same emotionally charged episode, and both can be right about what they ‘saw.’”
He adds, “So when my wife interprets my smirk as cynicism, she’s right: her explanation of my facial expression at that moment distorts her perception. But it’s also true that if she had interpreted my expression as empathy, I wouldn’t have had to sleep on the couch. It’s a paradox—the more we search for meaning in others’ emotions, the less accurately we remember them.”
Beyond Everyday Misunderstandings
The phenomenon goes far beyond everyday interpersonal misunderstandings. It is especially common among people with persistent or dysfunctional ways of interpreting emotions, such as those with anxiety disorders or psychological trauma. For example, anxious individuals tend to interpret others’ reactions negatively, which can permanently color their perception of feelings and intentions, reinforcing mistaken beliefs even in the face of contrary evidence.
Another area where these findings are relevant is eyewitness memory. A witness to a violent crime, for instance, may attribute anger to the perpetrator—a perception that, according to researchers, will influence their memory of the criminal’s facial expression.
The Experiment: How Biases Affect Memory
In the experiment, researchers showed participants digitally altered photos of faces displaying ambiguous emotions. Respondents were asked to think of these faces as either angry or happy. Then, participants watched the facial expressions slowly change from angry to happy and were asked to identify the photo they had originally seen.
Initial interpretations influenced memory: faces first interpreted as angry were more often remembered as angry than those initially seen as happy.
Our Bodies Mirror Our Biases
Researchers were especially interested in how ambiguous facial expressions could be perceived so differently. By measuring subtle electrical signals from facial muscles, they found that participants mimicked the emotions they had previously assigned to the ambiguous faces when viewing them again. In other words, when looking at a face they once thought was angry, people were more likely to display angry expressions themselves, compared to those who had initially interpreted the same face as happy.
These are largely automatic processes—this kind of facial mimicry reflects how people perceived the ambiguous face and shows that participants literally saw different expressions. Professor Winkielman notes, “We discovered that our body is an interface—a place where thoughts and perceptions meet. Our research supports the growing field of ‘embodied cognition’ and ‘embodied emotions.’ Our physical self is closely intertwined with how and what we think and feel.”