New memories such as the cat-elephant face are stored in an available pattern recognizer. The hippocampus plays a role in this process, and we’ll discuss what is known about the actual biological mechanisms in the following chapter. For the purposes of our neocortex model, it is sufficient to say that patterns that are not otherwise recognized are stored as new patterns and are appropriately connected to the lower-level patterns that form them. The cat-elephant face, for example, will be stored in several different ways: The novel arrangement of facial parts will be stored as well as contextual memories that include the artist, the situation, and perhaps the fact that we laughed when we first saw it.
Memories that are successfully recognized may also result in the creation of a new pattern to achieve greater redundancy. If patterns are not perfectly recognized, they are likely to be stored as reflecting a different perspective of the item that was recognized.
What, then, is the overall method for determining what patterns get stored? In mathematical terms, the problem can be stated as follows: Using the available limits of pattern storage, how do we optimally represent the input patterns that have thus far been presented? While it makes sense to allow for a certain amount of redundancy, it would not be practical to fill up the entire available storage area (that is, the entire neocortex) with repeated patterns, as that would not allow for a sufficient diversity of patterns. A pattern such as the [E] phoneme in spoken words is something we have experienced countless times. It is a simple pattern of sound frequencies and it undoubtedly enjoys significant redundancy in our neocortex. We could fill up our entire neocortex with repeated patterns of the [E] phoneme. There is a limit, however, to useful redundancy, and a common pattern such as this clearly has reached it.
There is a mathematical solution to this optimization problem called linear programming, which solves for the best possible allocation of limited resources (in this case, a limited number of pattern recognizers) that would represent all of the cases on which the system has trained. Linear programming is designed for systems with one-dimensional inputs, which is another reason why it is optimal to represent the input to each pattern recognition module as a linear string of inputs. We can use this mathematical approach in a software system, and though an actual brain is further constrained by the physical connections it has available that it can adapt between pattern recognizers, the method is nonetheless similar.
An important implication of this optimal solution is that experiences that are routine are recognized but do not result in a permanent memory’s being made. With regard to my walk, I experienced millions of patterns at every level, from basic visual edges and shadings to objects such as lampposts and mailboxes and people and animals and plants that I passed. Almost none of what I experienced was unique, and the patterns that I recognized had long since reached their optimal level of redundancy. The result is that I recall almost nothing from this walk. The few details that I do remember are likely to get overwritten with new patterns by the time I take another few dozen walks—except for the fact that I have now memorialized this particular walk by writing about it.
One important point that applies to both our biological neocortex and attempts to emulate it is that it is difficult to learn too many conceptual levels simultaneously. We can essentially learn one or at most two conceptual levels at a time. Once that learning is relatively stable, we can go on to learn the next level. We may continue to fine-tune the learning in the lower levels, but our learning focus is on the next level of abstraction. This is true at both the beginning of life, as newborns struggle with basic shapes, and later in life, as we struggle to learn new subject matter, one level of complexity at a time. We find the same phenomenon in machine emulations of the neocortex. However, if they are presented increasingly abstract material one level at a time, machines are capable of learning just as humans do (although not yet with as many conceptual levels).
The output of a pattern can feed back to a pattern at a lower level or even to the pattern itself, giving the human brain its powerful recursive ability. An element of a pattern can be a decision point based on another pattern. This is especially useful for lists that compose actions—for example, getting another tube of toothpaste if the current one is empty. These conditionals exist at every level. As anyone who has attempted to program a procedure on a computer knows, conditionals are vital to describing a course of action.
The Language of Thought
The dream acts as a safety-valve for the over-burdened brain.
Brain: an apparatus with which we think we think.
To summarize what we’ve learned so far about the way the neocortex works, please refer to the diagram of the neocortical pattern recognition module on page 42.
a) Dendrites enter the module that represents the pattern. Even though patterns may seem to have two- or three-dimensional qualities, they are represented by a one-dimensional sequence of signals. The pattern must be present in this (sequential) order for the pattern recognizer to be able to recognize it. Each of the dendrites is connected ultimately to one or more axons of pattern recognizers at a lower conceptual level that have recognized a lower-level pattern that constitutes part of this pattern. For each of these input patterns, there may be many lower-level pattern recognizers that can generate the signal that the lower-level pattern has been recognized. The necessary threshold to recognize the pattern may be achieved even if not all of the inputs have signaled. The module computes the probability that the pattern it is responsible for is present. This computation considers the “importance” and “size” parameters (see [f] below).
Note that some of the dendrites transmit signals into the module and some out of the module. If all of the input dendrites to this pattern recognizer are signaling that their lower-level patterns have been recognized except for one or two, then this pattern recognizer will send a signal down to the pattern recognizer(s) recognizing the lower-level patterns that have not yet been recognized, indicating that there is a high likelihood that that pattern will soon be recognized and that lower-level recognizer(s) should be on the lookout for it.
b) When this pattern recognizer recognizes its pattern (based on all or most of the input dendrite signals being activated), the axon (output) of this pattern recognizer will activate. In turn, this axon can connect to an entire network of dendrites connecting to many higher-level pattern recognizers that this pattern is input to. This signal will transmit magnitude information so that the pattern recognizers at the next higher conceptual level can consider it.