“And What Do You See?”: What Fundamental Differences in Perception Tell Us

Vision researchers quickly responded to the viral photo of the dress, noting the optical ambiguity of the lighting. If a person’s visual system interpreted the photo as being taken indoors under warm light, the dress appeared blue and black; if seen as outdoors, it looked white and gold. That same spring, at the annual Vision Sciences Society conference, a real version of the dress (for reference, it’s blue and black) was displayed under different lighting to show how dramatically its appearance changed. But none of this explains how different people’s perception automatically determined the lighting conditions (one predictive factor may be a person’s typical wake-up time: “night owls,” for example, spend more time in warm, artificial light).

Whatever the explanation, it’s remarkable that such a basic difference in visual perception could go unnoticed. Until the dress phenomenon (#TheDress), no one—not even vision scientists—suspected such contradictions in perception existed. Even more surprising is how easily these contradictions can be revealed. In the case of the dress, all it took was the question, “What color is it?” If we were so blind to such obvious differences in visual experience, how many more contradictions remain to be discovered, if only we know where to look and what questions to ask?

Hidden Differences in Perception: The Case of Aphantasia

Take the case of Blake Ross, co-founder of the Firefox browser. For the first three decades of his life, Blake considered his subjective experience completely ordinary. Why wouldn’t he? Until one day, he read a popular science article about people who lack the ability to see mental images. While most people can effortlessly form visual pictures in their mind’s eye, some cannot—this phenomenon was first documented in the 1800s but only recently named aphantasia. Ross realized from the article that he had it. His reaction was striking:

“Imagine your phone flashes a breaking news alert: ‘Washington scientists discover a man without a tail. Well, then what are we?’”

Ross began asking his friends and soon discovered that what he thought was normal—his inability to see images—was not. “I have never visualized anything in my life,” Ross wrote in Vox in 2016. “I can’t ‘see’ my father’s face or a bouncing blue ball, recall my childhood bedroom, or the run I took 10 minutes ago… I’m 30 years old, and I never knew people could do this. It blows my mind!”

Such hidden differences are accompanied by a sense of wonder. We’re attached to the idea of a certain order in things because our worldview depends on it. Meeting someone for whom the world works differently (even in something as simple as the color of a dress) means realizing our way of perceiving might be “wrong.” And if we can’t be sure about the color of a dress, what else might we be mistaken about? Similarly, for someone with aphantasia, learning that others have vivid mental images reveals a gulf between their perception and that of most people.

Why Studying Subtle Differences Matters

Studying these subtle details can enrich our scientific understanding of consciousness. For example, research into how bedtime affects color perception was inspired not by late-night people, but by a group of internet enthusiasts discussing their impressions of the two-tone dress. Research into aphantasia helps us understand how people without mental imagery can achieve the same goals (like recalling visual details of their living room) without using mental pictures. There are many such examples. Understanding these differences is likely a moral imperative as well, as it helps us appreciate the variety of human experience and empathize with those differences. The idea that someone might react differently to a situation not just because of different opinions or experiences, but because their perception is fundamentally different, is sobering.

For much of my scientific career, I didn’t think much about individual differences. Like most cognitive scientists, I manipulated variables to see how they affected group averages. My research focused on how language complements human cognition and perception. For example, I studied whether learning new object names affects how people perceive, remember, and categorize them. These were typical group effect studies, comparing how people respond to different stimuli. As usual, people responded differently, but we focused on average responses.

For example, when hearing the word “green,” people can distinguish subtle differences between more and less saturated green samples. When we ask participants to do a parallel verbal task that conflicts with an image—like grouping objects by size or color—most have difficulty. But “most” isn’t “all.” Could it be that language helps some people distinguish colors and categorize objects, but not others? This led us to search for other hidden perceptual differences, like aphantasia.

Inner Speech: Another Universal, Yet Variable, Human Experience

We were particularly interested in the universal human phenomenon of inner speech. Most people report having an inner voice. For example, 83% (3,445 out of 4,145 in our sample) “agree” or “strongly agree” with the statement, “When I read, I hear a voice in my head.” Similarly, 80% “agree” or “strongly agree” with, “I think about problems in the form of a dialogue with myself.” This rises to 85% for social issues: “When I think about a social problem, I often internally talk it through with myself.”

But even 85% isn’t everyone. What about those who disagree? Some report having an inner voice only in certain situations. When it comes to reading, some say they hear an inner voice only if they slow down or read something difficult. A small number (2-5%) say they never hear an inner voice. Like people with aphantasia, who think visual images are just a figure of speech, people with anendophasia (a term coined by me and Joan Nedergaard for the absence of inner speech) believe that “inner monologues,” so common in TV shows, are just a cinematic device, not something people actually experience. People with anendophasia don’t replay past conversations in their heads, and while they generally know what they want to say, they don’t know exactly what until they start speaking.

It’s tempting to think there’s a trade-off between thinking in words and thinking in images. Take the popular idea of different learning styles—mainly verbal or visual (a classification that is, by the way, incorrect). When it comes to visual imagery and inner speech, we find a moderate positive correlation between vividness of imagery and presence of inner speech. People with more vivid imagery tend to have more inner speech, and vice versa—those without inner speech report few images.

This raises the question: what do their thoughts feel like? When we asked, we usually got vague answers like, “I think in ideas,” or “I think in concepts.” We have many ways to describe perceptual qualities (especially visual), and of course, we can use language to talk about language. So it’s not surprising that people struggle to describe thoughts without perceptual or linguistic traces. But difficulty expressing such thoughts doesn’t make them less real. It just means people have to work harder to understand themselves.

Other Hidden Differences: Synesthesia, Eigengrau, and Memory

Differences in visual imagery and inner speech are just the tip of the iceberg. Other hidden differences include phenomena like synesthesia (from Greek, “union of senses”), where people hear light or taste sounds, and Eigengrau (from German, “intrinsic gray”), the color we see when we close our eyes. But not everyone sees Eigengrau. About 10% of respondents say their experience is nothing like Eigengrau; instead, they see colorful patterns or visual static, like an untuned analog TV.

Our memory is subject to much greater differences than we might expect. In 2015, psychologist Daniela Palombo and colleagues published a study describing Severely Deficient Autobiographical Memory (SDAM). A person with SDAM may know they traveled to Italy five years ago but can’t recall the event from a first-person perspective—they can’t “mentally time travel” as most of us take for granted. As with other hidden differences, these people often don’t realize they’re unusual. As Claudia Hammond wrote for the BBC about Susie McKinnon, one of the first people identified with SDAM, she always “assumed that when people told detailed stories about their past, they were just making up details to entertain others.”

Why Are These Differences Hidden?

What makes differences in imagery, inner speech, synesthesia, and memory so hidden? It’s tempting to think it’s because we can’t observe them directly. We can see if someone runs fast. But with direct access only to our own reality, how can we know what someone else imagines when they think of an apple, or whether they hear a voice when reading? Still, while we can’t directly experience another’s reality, we can compare it to ours by talking about it. Sometimes it’s simple: for #TheDress, we just had to ask each other what colors we saw. We can also ask if letters always appear in color (grapheme-color synesthetes will say “yes,” others “no”).

People without mental imagery will tell you they can’t picture an apple, and people without inner speech will say they don’t have silent conversations with themselves. In fact, it’s not hard to discover these differences if we start to study them systematically.

Paradoxically, while language lets us compare experiences and discover differences in our subjective lives, its tendency toward abstraction can also make us overlook these differences, since the same word can mean many things. We use “imagine” to mean forming a mental image, but also for more abstract actions, like imagining a hypothetical future. For someone with aphantasia, it’s logical not to realize that sometimes people use “imagine” to mean forming perceptual mental images.

Can We Trust Self-Reports?

Most of our understanding of these subtle differences relies on self-reports. Can we trust them? Modern psychology is skeptical of self-reports, and I inherited this skepticism in my academic training. Recent reports of significant individual differences in imagery and inner speech are often met with skepticism. How do we know these differences reflect something real? Can we really just take people at their word when they say they have no inner voice?

But before tackling the complex question of trusting people’s reports about their imagery and inner speech, let’s consider simpler examples. When someone says they don’t like cauliflower, they’re reporting a subjective experience, and others tend to believe them. But we can easily run an experiment to see how likely they are to choose cauliflower when given alternatives. It would be curious if someone claims not to like cauliflower but eats it at every opportunity. Of course, discrepancies between stated and actual preferences exist. Many scientists have built careers studying such gaps. For example, if cauliflower is socially approved in a culture, people may say they like it even if they don’t. Conversely, someone might eat cauliflower just to avoid offending a host. Such situations require caution in interpreting both stated and revealed preferences, but in ordinary circumstances, trusting people’s word about their preferences is a good guide to understanding their behavior.

Consider another case. You work in an office, and your coworker says they’re cold when the thermostat is set to 72°F (22°C). Do you believe them or say, “But 72 degrees is normal room temperature. How can you be cold?” Suppose we measure skin temperature, body temperature, even do an fMRI scan showing insula activation. None of these measurements can prove the person isn’t cold. None can refute their self-report. If someone has hypothermia, objective measures are appropriate, but if the goal is simply to understand what someone feels, self-reports are superior to objective measures.

The same logic applies to other subjective states, like loneliness, pain, and awe. To measure loneliness, it’s not enough to count how many people someone interacts with, because one person’s active social life may be another’s loneliness. We can detect a flu epidemic with objective tests, but to diagnose a “loneliness epidemic,” we must consider whether people feel lonely. That’s why, despite all our physiological measurement technology, when it comes to pain, we still rely on pain scales—a simple form of self-report. If we take introspective judgments seriously for preferences, emotions, and pain, why should we be more skeptical about them for phenomenological differences like imagery and inner speech?

One reason is that some things we can reliably self-analyze, and others we can’t. Maybe we can reliably report “basic” states like pain or whether we like cauliflower (though even here, self-reflection can vary), but in other cases, introspection fails. For example, most people think they’re above-average drivers—one of many examples of the “Lake Wobegon effect.” We can also be wrong in the other direction. In typical implicit learning studies, participants see sequences of flashing lights, sounds, or shapes that follow a rule. Later, they must judge if new sequences follow the same rule. Participants often feel they’re just guessing, but their performance can be well above chance, showing they did learn. In such cases, “wrong” self-reports are still informative: they tell us about the person’s subjective reality (they think they’re 80th percentile drivers, they think they’re guessing, they think they didn’t learn, even if they did). But these self-reports don’t reflect objective reality and are poor predictors of what someone does or can do.

Dreams and the Reliability of Introspection

Finally, we must consider dreams. In a 1958 study, Fernando Tapia and colleagues reported that only 9% of respondents saw dreams in color. Other surveys at the time also showed low percentages. A decade later, most people reported seeing color dreams. Philosopher Eric Schwitzgebel found several explanations for this shift. One likely version: the world of black-and-white photography and TV changed the world of human dreams. With the rise of color TV, color returned to people’s dreams (returned, because earlier accounts suggest people didn’t typically dream in black and white).

The problem is, there’s no reason to think TV should have such a strong effect on our dream phenomenology. After all, the world never stopped being in color. The alternative, Schwitzgebel argues, is that “at least some people must be seriously mistaken about their dreams.” Our ability to report dream content may simply be unreliable. And without objective measures for subjective reports, we can’t know if these reports reflect any reality, subjective or otherwise. So why is there any consistency in people’s reports at a given time? Perhaps because, lacking good access to their phenomenal states, people choose the answer that seems most reasonable. In the 1950s, the dominant view was that dreams weren’t in color, so people repeated that view. The same happened when the dominant view changed. Schwitzgebel argues that neither case reflects the “correct” phenomenology, because our introspection may be unreliable when it comes to the color of our dreams.

If reports about phenomenal states like imagery and inner speech are like reports about dreams, we have every reason to be skeptical that differences in introspection reflect real differences in people’s actual experience. If they’re more like reports about preferences and emotions, we can (mostly) take people at their word. But even then, we must consider social pressures to respond a certain way. If vivid imagery were required for art school admission, we shouldn’t be surprised if all aspiring artists claimed to have vivid images. If hearing a voice while reading were seen as a sign of mental illness, people would be less likely to admit it.

How Can We Validate Self-Reports?

There are several ways to validate self-analysis. First, we should demonstrate consistency. If people say one day they always have inner speech, and the next day say they never do, that’s a problem. As it turns out, people’s reports are very consistent. Inner speech questionnaires given months apart show high correlation. (At the same time, Russell Hurlburt’s work, which samples people’s thoughts at random times during the day, shows people overestimate how much their thinking takes the form of inner speech.)

We can also see if differences in reported phenomenology predict differences in behavior. With dreams, this isn’t possible, but we can make specific predictions about behavioral consequences of more or less visual imagery and inner speech, based on existing theories. Differences in self-reported phenomenology can be linked to differences in objective behavior. Those with less inner speech have more trouble remembering word lists; those with less visual imagery report fewer visual details when describing past events.

There are also differences in more automatic physiological responses. More light entering the pupil causes it to constrict. But simply showing something bright, like sunlight, also causes (a smaller but measurable) constriction. Aphantasics show typical pupil responses to real changes in light, but their pupils don’t change to imagined light. Meanwhile, many hypothetical behavioral differences aren’t observed, because people seem to compensate, for example, by finding ways to remember detailed visual content without using explicit images. Such compensation can be useful. People with poor autobiographical memory find other ways to remember information, which may help prevent some cognitive decline with age.

Another way to validate is to ask if there are neural and physiological correlates of phenomenal differences. If differences in reported imagery were just confabulations or people telling researchers what they think they want to hear, it would be surprising if they had different brain connectivity and functional activation as measured by fMRI. But that’s exactly what we find. Fraser Milton and colleagues scanned groups identified as aphantasics and hyperphantasics (people with unusually vivid imagery). When asked to lie in a scanner and look at a cross on a screen, the hyperphantasic group showed greater connectivity between the prefrontal cortex and occipital visual network compared to the aphantasic group. Participants were also asked to look at and imagine various famous people and places. The difference in activation between perception and imagination (in the left anterior parietal area) was greater in hyperphantasics than in aphantasics. Those with typical imagination were in between on most measures. Less is known about neural correlates of inner speech differences. In a study presented at the Society for the Neurobiology of Language in 2023, Huichao Yang and colleagues found a link between the amount of inner speech people reported and the level of functional connectivity in the resting-state language network.

Finally, while we can’t know what it’s like to be someone else, we can compare how our phenomenology changes over time. There are many studies of people with brain injuries who lose visual imagery, and in some cases, inner speech. It’s much harder to dismiss self-reports from people who say they used to be able to imagine things, but now can’t (especially when supported by clear differences in objective behavior).

Why This Matters: The Scientific and Moral Imperative

The post that introduced the world to #TheDress had a second part. The author wrote:

“This matters because I feel like I’m going crazy.”

The idea that the same image can look different to different people is unsettling because it threatens our confidence that the world is as we perceive it. When an aphantasic learns that others spontaneously experience mental images, they realize they’re encountering a daily reality for many people they never even suspected existed. It’s clearly destabilizing.

And yet, there is both a scientific and moral imperative to study the diversity of our phenomenology. Scientifically, it prevents us from claiming that the experience of the majority (or the scientist) is the experience of all. Morally, it encourages us to go beyond the ancient advice to “know thyself” and focus on knowing others. And that requires being open to the possibility that their experience may be very different from our own.