Autoassociation and Invariance
In the previous chapter I discussed how we can recognize a pattern even if the entire pattern is not present, and also if it is distorted. The first capability is called autoassociation: the ability to associate a pattern with a part of itself. The structure of each pattern recognizer inherently supports this capability.
As each input from a lower-level pattern recognizer flows up to a higher-level one, the connection can have a “weight,” indicating how important that particular element in the pattern is. Thus the more significant elements of a pattern are more heavily weighted in considering whether that pattern should trigger as “recognized.” Lincoln’s beard, Elvis’s sideburns, and Einstein’s famous tongue gesture are likely to have high weights in the patterns we’ve learned about the appearance of these iconic figures. The pattern recognizer computes a probability that takes the importance parameters into account. Thus the overall probability is lower if one or more of the elements is missing, though the threshold of recognition may nonetheless be met. As I pointed out, the computation of the overall probability (that the pattern is present) is more complicated than a simple weighted sum in that the size parameters also need to be considered.
If the pattern recognizer has received a signal from a higher-level recognizer that its pattern is “expected,” then the threshold is effectively lowered (that is, made easier to achieve). Alternatively, such a signal may simply add to the total of the weighted inputs, thereby compensating for a missing element. This happens at every level, so that a pattern such as a face that is several levels up from the bottom may be recognized even with multiple missing features.
The ability to recognize patterns even when aspects of them are transformed is called feature invariance, and is dealt with in four ways. First, there are global transformations that are accomplished before the neocortex receives sensory data. We will discuss the voyage of sensory data from the eyes, ears, and skin in the section “The Sensory Pathway” on page 94.
The second method takes advantage of the redundancy in our cortical pattern memory. Especially for important items, we have learned many different perspectives and vantage points for each pattern. Thus many variations are separately stored and processed.
The third and most powerful method is the ability to combine two lists. One list can have a set of transformations that we have learned may apply to a certain category of pattern; the cortex will apply this same list of possible changes to another pattern. That is how we understand such language phenomena as metaphors and similes.
For example, we have learned that certain phonemes (the basic sounds of language) may be missing in spoken speech (for example, “goin’”). If we then learn a new spoken word (for example, “driving”), we will be able to recognize that word if one of its phonemes is missing even if we have never experienced that word in that form before, because we have become familiar with the general phenomenon of certain phonemes being omitted. As another example, we may learn that a particular artist likes to emphasize (by making larger) certain elements of a face, such as the nose. We can then identify a face with which we are familiar to which that modification has been applied even if we have never seen that modification on that face. Certain artistic modifications emphasize the very features that are recognized by our pattern recognition–based neocortex. As mentioned, that is precisely the basis of caricature.
The fourth method derives from the size parameters that allow a single module to encode multiple instances of a pattern. For example, we have heard the word “steep” many times. A particular pattern recognition module that is recognizing this spoken word can encode these multiple examples by indicating that the duration of [E] has a high expected variability. If all the modules for words including [E] share a similar phenomenon, that variability could be encoded in the models for [E] itself. However, different words incorporating [E] (or many other phonemes) may have different amounts of expected variability. For example, the word “peak” is likely not to have the [E] phoneme as drawn out as in the word “steep.”
Learning
Are we not ourselves creating our successors in the supremacy of the earth? Daily adding to the beauty and delicacy of their organization, daily giving them greater skill and supplying more and more of that self-regulating self-acting power which will be better than any intellect?
The principal activities of brains are making changes in themselves.
So far we have examined how we recognize (sensory and perceptual) patterns and recall sequences of patterns (our memory of things, people, and events). However, we are not born with a neocortex filled with any of these patterns. Our neocortex is virgin territory when our brain is created. It has the capability of learning and therefore of creating connections between its pattern recognizers, but it gains those connections from experience.
This learning process begins even before we are born, occurring simultaneously with the biological process of actually growing a brain. A fetus already has a brain at one month, although it is essentially a reptile brain, as the fetus actually goes through a high-speed re-creation of biological evolution in the womb. The natal brain is distinctly a human brain with a human neocortex by the time it reaches the third trimester of pregnancy. At this time the fetus is having experiences, and the neocortex is learning. She can hear sounds, especially her mother’s heartbeat, which is one likely reason that the rhythmic qualities of music are universal to human culture. Every human civilization ever discovered has had music as part of its culture, which is not the case with other art forms, such as pictorial art. It is also the case that the beat of music is comparable to our heart rate. Music beats certainly vary—otherwise music would not keep our interest—but heartbeats vary also. An overly regular heartbeat is actually a symptom of a diseased heart. The eyes of a fetus are partially open twenty-six weeks after conception, and are fully open most of the time by twenty-eight weeks after conception. There may not be much to see inside the womb, but there are patterns of light and dark that the neocortex begins to process.
So while a newborn baby has had a bit of experience in the womb, it is clearly limited. The neocortex may also learn from the old brain (a topic I discuss in chapter 5), but in general at birth the child has a lot to learn—everything from basic primitive sounds and shapes to metaphors and sarcasm.
Learning is critical to human intelligence. If we were to perfectly model and simulate the human neocortex (as the Blue Brain Project is attempting to do) and all of the other brain regions that it requires to function (such as the hippocampus and thalamus), it would not be able to do very much—in the same way that a newborn infant cannot do much (other than to be cute, which is definitely a key survival adaptation).
Learning and recognition take place simultaneously. We start learning immediately, and as soon as we’ve learned a pattern, we immediately start recognizing it. The neocortex is continually trying to make sense of the input presented to it. If a particular level is unable to fully process and recognize a pattern, it gets sent to the next higher level. If none of the levels succeeds in recognizing a pattern, it is deemed to be a new pattern. Classifying a pattern as new does not necessarily mean that every aspect of it is new. If we are looking at the paintings of a particular artist and see a cat’s face with the nose of an elephant, we will be able to identify each of the distinctive features but will notice that this combined pattern is something novel, and are likely to remember it. Higher conceptual levels of the neocortex, which understand context—for example, the circumstance that this picture is an example of a particular artist’s work and that we are attending an opening of a showing of new paintings by that artist—will note the unusual combination of patterns in the cat-elephant face but will also include these contextual details as additional memory patterns.