As we experience our own thoughts and memories, we “know” what they mean, but they do not exist as readily explainable thoughts and recollections. If we want to share them with others, we need to translate them into language. This task is also accomplished by the neocortex, using pattern recognizers trained with patterns that we have learned for the purpose of using language. Language is itself highly hierarchical and evolved to take advantage of the hierarchical nature of the neocortex, which in turn reflects the hierarchical nature of reality. The innate ability of humans to learn the hierarchical structures in language that Noam Chomsky wrote about reflects the structure of the neocortex. In a 2002 paper he coauthored, Chomsky cites the attribute of “recursion” as accounting for the unique language faculty of the human species.4 Recursion, according to Chomsky, is the ability to put together small parts into a larger chunk, and then use that chunk as a part in yet another structure, and to continue this process iteratively. In this way we are able to build the elaborate structures of sentences and paragraphs from a limited set of words. Although Chomsky was not explicitly referring here to brain structure, the capability he is describing is exactly what the neocortex does.
Lower species of mammals largely use up their neocortex with the challenges of their particular lifestyles. The human species acquired additional capacities by having grown substantially more cortex to handle spoken and written language. Some people have learned such skills better than others. If we have told a particular story many times, we will begin to actually learn the sequence of language that describes the story as a series of separate sequences. Even in this case our memory is not a strict sequence of words, but rather of language structures that we need to translate into specific word sequences each time we deliver the story. That is why we tell a story a bit differently each time we share it (unless we learn the exact word sequence as a pattern).
For each of these descriptions of specific thought processes, we also need to consider the issue of redundancy. As I mentioned, we don’t have a single pattern representing the important entities in our lives, whether those entities constitute sensory categories, language concepts, or memories of events. Every important pattern—at every level—is repeated many times. Some of these recurrences represent simple repetitions, whereas many represent different perspectives and vantage points. This is a principal reason why we can recognize a familiar face from various orientations and under a range of lighting conditions. Each level up the hierarchy has substantial redundancy, allowing sufficient variability that is consistent with that concept.
So if we were to imagine examining your neocortex when you were looking at a particular loved one, we would see a great many firings of the axons of the pattern recognizers at every level, from the basic level of primitive sensory patterns up to many different patterns representing that loved one’s image. We would also see massive numbers of firings representing other aspects of the situation, such as that person’s movements, what she is saying, and so on. So if the experience seems much richer than just an orderly trip up a hierarchy of features, it is.
A computer simulation of the firings of many simultaneous pattern recognizers in the neocortex.
But the basic mechanism of going up a hierarchy of pattern recognizers in which each higher conceptual level represents a more abstract and more integrated concept remains valid. The flow of information downward is even greater, as each activated level of recognized pattern sends predictions to the next lower-level pattern recognizer of what it is likely to be encountering next. The apparent lushness of human experience is a result of the fact that all of the hundreds of millions of pattern recognizers in our neocortex are considering their inputs simultaneously.
In chapter 5 I’ll discuss the flow of information from touch, vision, hearing, and other sensory organs into the neocortex. These early inputs are processed by cortical regions that are devoted to relevant types of sensory input (although there is enormous plasticity in the assignment of these regions, reflecting the basic uniformity of function in the neocortex). The conceptual hierarchy continues above the highest concepts in each sensory region of the neocortex. The cortical association areas integrate input from the different sensory inputs. When we hear something that perhaps sounds like our spouse’s voice, and then see something that is perhaps indicative of her presence, we don’t engage in an elaborate process of logical deduction; rather, we instantly perceive that our spouse is present from the combination of these sensory recognitions. We integrate all of the germane sensory and perceptual cues—perhaps even the smell of her perfume or his cologne—as one multilevel perception.
At a conceptual level above the cortical sensory association areas, we are capable of dealing with—perceiving, remembering, and thinking about—even more abstract concepts. At the highest level we recognize patterns such as that’s funny, or she’s pretty, or that’s ironic, and so on. Our memories include these abstract recognition patterns as well. For example, we might recall that we were taking a walk with someone and that she said something funny, and we laughed, though we may not remember the actual joke itself. The memory sequence for that recollection has simply recorded the perception of humor but not the precise content of what was funny.
In the previous chapter I noted that we can often recognize a pattern even though we don’t recognize it well enough to be able to describe it. For example, I believe I could pick out a picture of the woman with the baby carriage whom I saw earlier today from among a group of pictures of other women, despite the fact that I am unable to actually visualize her and cannot describe much specific about her. In this case my memory of her is a list of certain high-level features. These features do not have language or image labels attached to them, and they are not pixel images, so while I am able to think about her, I am unable to describe her. However, if I am presented with a picture of her, I can process the image, which results in the recognition of the same high-level features that were recognized the first time I saw her. I would be able to thereby determine that the features match and thus confidently pick out her picture.
Even though I saw this woman only once on my walk, there are probably already multiple copies of her pattern in my neocortex. However, if I don’t think about her for a given period of time, then these pattern recognizers will become reassigned to other patterns. That is why memories grow dimmer with time: The amount of redundancy becomes reduced until certain memories become extinct. However, now that I have memorialized this particular woman by writing about her here, I probably won’t forget her so easily.