It should be noted, before we further consider the structure of the neocortex, that it is important to model systems at the right level. Although chemistry is theoretically based on physics and could be derived entirely from physics, this would be unwieldy and infeasible in practice, so chemistry has established its own rules and models. Similarly, we should be able to deduce the laws of thermodynamics from physics, but once we have a sufficient number of particles to call them a gas rather than simply a bunch of particles, solving equations for the physics of each particle interaction becomes hopeless, whereas the laws of thermodynamics work quite well. Biology likewise has its own rules and models. A single pancreatic islet cell is enormously complicated, especially if we model it at the level of molecules; modeling what a pancreas actually does in terms of regulating levels of insulin and digestive enzymes is considerably less complex.
The same principle applies to the levels of modeling and understanding in the brain. It is certainly a useful and necessary part of reverse-engineering the brain to model its interactions at the molecular level, but the goal of the effort here is essentially to refine our model to account for how the brain processes information to produce cognitive meaning.
American scientist Herbert A. Simon (1916–2001), who is credited with cofounding the field of artificial intelligence, wrote eloquently about the issue of understanding complex systems at the right level of abstraction. In describing an AI program he had devised called EPAM (elementary perceiver and memorizer), he wrote in 1973, “Suppose you decided that you wanted to understand the mysterious EPAM program that I have. I could provide you with two versions of it. One would be…the form in which it was actually written—with its whole structure of routines and subroutines…. Alternatively, I could provide you with a machine-language version of EPAM after the whole translation had been carried out—after it had been flattened so to speak…. I don’t think I need argue at length which of these two versions would provide the most parsimonious, the most meaningful, the most lawful description…. I will not even propose to you the third…of providing you with neither program, but instead with the electromagnetic equations and boundary conditions that the computer, viewed as a physical system, would have to obey while behaving as EPAM. That would be the acme of reduction and incomprehensibility.”3
There are about a half million cortical columns in a human neocortex, each occupying a space about two millimeters high and a half millimeter wide and containing about 60,000 neurons (resulting in a total of about 30 billion neurons in the neocortex). A rough estimate is that each pattern recognizer within a cortical column contains about 100 neurons, so there are on the order of 300 million pattern recognizers in total in the neocortex.
As we consider how these pattern recognizers work, let me begin by saying that it is difficult to know precisely where to begin. Everything happens simultaneously in the neocortex, so there is no beginning and no end to its processes. I will frequently need to refer to phenomena that I have not yet explained but plan to come back to, so please bear with these forward references.
Human beings have only a weak ability to process logic, but a very deep core capability of recognizing patterns. To do logical thinking, we need to use the neocortex, which is basically a large pattern recognizer. It is not an ideal mechanism for performing logical transformations, but it is the only facility we have for the job. Compare, for example, how a human plays chess to how a typical computer chess program works. Deep Blue, the computer that defeated Garry Kasparov, the human world chess champion, in 1997 was capable of analyzing the logical implications of 200 million board positions (representing different move-countermove sequences) every second. (That can now be done, by the way, on a few personal computers.) Kasparov was asked how many positions he could analyze each second, and he said it was less than one. How is it, then, that he was able to hold up to Deep Blue at all? The answer is the very strong ability humans have to recognize patterns. However, we need to train this facility, which is why not everyone can play master chess.
Kasparov had learned about 100,000 board positions. That’s a real number—we have established that a human master in a particular field has mastered about 100,000 chunks of knowledge. Shakespeare composed his plays with 100,000 word senses (employing about 29,000 distinct words, but using most of them in multiple ways). Medical expert systems that have been built to represent the knowledge of a human medical physician have shown that a typical human medical specialist has mastered about 100,000 concepts in his or her domain. Recognizing a chunk of knowledge from this store is not straightforward, as a particular item will present itself a little bit differently each time it is experienced.
Armed with his knowledge, Kasparov looks at the chessboard and compares the patterns that he sees to all 100,000 board situations that he has mastered, and he does all 100,000 comparisons simultaneously. There is consensus on this point: All of our neurons are processing—considering the patterns—at the same time. That does not mean that they are all firing simultaneously (we would probably fall to the floor if that happened), but while doing their processing are considering the possibility of firing.
How many patterns can the neocortex store? We need to factor in the phenomenon of redundancy. The face of a loved one, for example, is not stored once but on the order of thousands of times. Some of these repetitions are largely the same image of the face, whereas most show different perspectives of it, different lighting, different expressions, and so on. None of these repeated patterns are stored as images per se (that is, as two-dimensional arrays of pixels). Rather, they are stored as lists of features where the constituent elements of a pattern are themselves patterns. We’ll describe below more precisely what these hierarchies of features look like and how they are organized.
If we take the core knowledge of an expert as consisting of about 100,000 “chunks” of knowledge (that is, patterns) with a redundancy estimate of about 100 to 1, that gives us a requirement of 10 million patterns. This core expert knowledge is built on more general and extensive professional knowledge, so we can increase the order of magnitude of patterns to about 30 to 50 million. Our everyday “commonsense” knowledge as a human being is even greater; “street smarts” actually require substantially more of our neocortex than “book smarts.” Including this brings our estimate to well over 100 million patterns, taking into account the redundancy factor of about 100. Note that the redundancy factor is far from fixed—very common patterns will have a redundancy factor well into the thousands, whereas a brand-new phenomenon may have a redundancy factor of less than 10.
As I will discuss below, our procedures and actions also comprise patterns and are likewise stored in regions of the cortex, so my estimate of the total capacity of the human neocortex is on the order of low hundreds of millions of patterns. This rough tally correlates well with the number of pattern recognizers that I estimated above at about 300 million, so it is a reasonable conclusion that the function of each neocortical pattern recognizer is to process one iteration (that is, one copy among the multiple redundant copies of most patterns in the neocortex) of a pattern. Our estimates of the number of patterns that a human brain is capable of dealing with (including necessary redundancy) and the number of physical pattern recognizers happen to be the same order of magnitude. It should be noted here that when I refer to “processing” a pattern, I am referring to all of the things we are able to do with a pattern: learn it, predict it (including parts of it), recognize it, and implement it (either by thinking about it further or through a pattern of physical movement).