When he reached home, he saw an engraving of St. Paul’s Cathedral that had been hanging on the wall for decades. He said he knew someone had given it to him but had forgotten what it depicted. He could produce an astonishingly accurate drawing, copying its every detail—including printing flaws! But even after he had done so, he still couldn’t say what it was. John could see perfectly clearly; he just didn’t know what he was seeing—which is why the flaws weren’t “flaws” for him.
John had been an avid gardener prior to his stroke. He walked out of his house and much to his wife’s surprise picked up a pair of shears and proceeded to trim the hedge effortlessly. However, when he tried to tidy up the garden, he often plucked the flowers from the ground because he couldn’t tell them from the weeds. Trimming the hedge, on the other hand, required only that John see where the unevenness was. No identification of objects was required. The distinction between seeing and knowing is illustrated well by John’s predicament.
Although an inability to know what he was looking at was John’s main problem, he had other subtler difficulties as well. For instance he had tunnel vision, often losing the proverbial forest for the trees. He could reach out and grab a cup of coffee when it was on an uncluttered table by itself, but got hopelessly muddled when confronted with a buffet service. Imagine his surprise when he discovered he had poured mayonnaise rather than cream into his coffee.
Our perception of the world ordinarily seems so effortless that we tend to take it for granted. You look, you see, you understand—it seems as natural and inevitable as water flowing downhill. Its only when something goes wrong, as in patients like John, that we realize how extraordinarily sophisticated it really is. Even though our picture of the world seems coherent and unified, it actually emerges from the activity those thirty (or more) different visual areas in the cortex, each of which mediates multiple subtle functions. Many of these areas are ones we share with other mammals but some of them “split” off at some point to become newly specialized modules in higher primates. Exactly how many of our visual areas are unique to humans isn’t clear. But a great deal more is known about them than about other higher brain regions such as the frontal lobes, which are involved in such things as morality, compassion, and ambition. A thorough understanding of how the visual system really works may therefore provide insights into the more general strategies the brain uses to handle information, including the ones that are unique to us.
A FEW YEARS ago I was at an after-dinner speech given by David Attenborough at the university aquarium in La Jolla, California, near where I work. Sitting next to me was a distinguished-looking man with a walrus moustache. After his fourth glass of wine he told me that he worked for the creation science institute in San Diego. I was very tempted to tell him that creation science is an oxymoron, but before I could do so he interrupted me to ask where I worked and what I was currently interested in.
“Autism and synesthesia these days. But I also study vision.”
“Vision? What’s there to study?”
“Well, what do you think goes on in your head when you look at something—that chair for example?”
“There is an optical image of the chair in my eye—on my retina. The image is transmitted along a nerve to the visual area of the brain and you see it. Of course, the image in the eye is upside down, so it has to be made upright again in the brain before you see it.”
His answer embodies a logical fallacy called the homunculus fallacy. If the image on the retina is transmitted to the brain and “projected” on some internal mental screen, then you would need some sort of “little man”—a homunculus—inside your head looking at the image and interpreting or understanding it for you. But how would the homunculus be able to understand the images flashing by on his screen? There would have to be another, even smaller chap looking at the image in his head—and so on. It is a situation of infinite regress of eyes, images, and little people, without really solving the problem of perception.
In order to understand perception, you need to first get rid of the notion that the image at the back of your eye simply gets “relayed” back to your brain to be displayed on a screen. Instead, you must understand that as soon as the rays of light are converted into neural impulses at the back of your eye, it no longer makes any sense to think of the visual information as being an image. We must think, instead, of symbolic descriptions that represent the scenes and objects that had been in the image. Say I wanted someone to know what the chair across the room from me looks like. I could take him there and point it out to him so he could see it for himself, but that isn’t a symbolic description. I could show him a photograph or a drawing of the chair, but that is still not symbolic because it bears a physical resemblance. But if I hand the person a written note describing the chair, we have crossed over into the realm of symbolic description: The squiggles of ink on the paper bear no physical resemblance to the chair; they merely symbolize it.
Analogously, the brain creates symbolic descriptions. It does not re-create the original image, but represents the various features and aspects of the image in totally new terms—not with squiggles of ink, of course, but in its own alphabet of nerve impulses. These symbolic encodings are created partly in your retina itself but mostly in your brain. Once there, they are parceled and transformed and combined in the extensive network of visual brain areas that eventually let you recognize objects. Of course, the vast majority of this processing goes on behind the scenes without entering your conscious awareness, which is why it feels effortless and obvious, as it did to my dinner companion.
I’ve been glibly dismissing the homunculus fallacy by pointing out the logical problem of infinite regress. But is there any direct evidence that it is in fact a fallacy?
First, what you see can’t just be the image on the retina because the retinal image can remain constant but your perception can change radically. If perception simply involves transmitting and displaying an image on an inner mental screen, how can this be true? Second, the converse is also true: The retinal image can change, yet your perception of the object remains stable. Third, despite appearances, perception takes time and happens in stages.
The first reason is the most easy to appreciate. It’s the basis of many visual illusions. A famous example is the Necker cube, discovered accidentally by the Swiss crystallographer Louis Albert Necker (Figure 2.1). He was gazing at a cuboid crystal through a microscope one day, and imagine his amazement when the crystal suddenly seemed to flip! Without visibly moving, it switched its orientation right in front of his very eyes. Was the crystal itself changing? To find out he drew a wire-frame cube on a scrap of paper and noticed that the drawing did the same thing. Conclusion: His perception was changing, not the crystal. You can try this on yourself. It is fun even if you have tried it dozens of times in the past. You will see that the drawing suddenly flips on you, and it’s partly—but only partly—under voluntary control. The fact that your perception of an unchanging image can change and flip radically is proof that perception must involve more than simply displaying an image in the brain. Even the simplest act of perception involves judgment and interpretation. Perception is an actively formed opinion of the world rather than a passive reaction to sensory input from it.
FIGURE 2.1 Skeleton outline drawing of a cube: You can see it in either of two different ways, as if it were above you or below you.
FIGURE 2.2 This picture has not been Photoshopped! It was taken with an ordinary camera from the special viewing point that makes the Ames room work. The fun part of this illusion comes when you have two people walk to opposite ends of the room: It looks for all the world as if they are standing just a few feet apart from each other and one of them has grown giant, with his head brushing the ceiling, while the other has shrunk to the size of a fairy.