How are insights generated? This is related to another problem. The brain has memory for sure, but does it have a processor?
Theory of neurocomputing
Hardware and software are just parts of another item – a whole computational system and it is obvious that a computer based on the Von Neumann architecture and the human brain are very different. Paradox is that computers were thoroughly developed for decades and we know them in details while evolution of the brain spans millions of years, we use it permanently, but have no clues about its operation. Substantial efforts were made during 19 and 20 centuries so as to fill this gap. Let's try to formulate explicitly the most basic principles of neurocomputing. First, we need to determine our goal. What is a neurocomputer? Any construct created of neural nets? Their range would be as wide as diversity of live nervous systems. Let's confine our interest to the human brain, but keep in mind that it isn't the top of perfectness. Maybe the brain of some animals is better in some aspect. Maybe there are better solutions unknown in the nature. The prototype is only a hint for the theory.
Let's formulate the answer at the very beginning, then explain it in details. Thorough consideration shows that the following 3 concepts are workable for both types of computing, but they have different implementation. Moreover, distribution of overall computing between memory, a processor, and software is different. A neurocomputer uses associative memory implemented as blocks of homogeneous neural nets. Even in the simplest form with 2 layers, they are already capable of some processing such as branching or simple arithmetic (addition and subtraction). That's it. We have processing without a processor. Add the difference in data representation. Computer memory stores data immediately. In neural nets, data is an image represented as a pattern of neural activity, but memory is kept in modifiable synapses. Nets don't store images immediately. They store the ability to generate particular images in response to particular keys. This entails another difference. In a computer, data is loaded from memory into a processor, undergo modification, then is stored back. In a neurocomputer, images are processed immediately in the memory. Elementary processor instructions are replaced by hard-wired ability of a particular block of memory to perform particular image transformations. This ability is defined by local interconnections within the same layer or between different layers of the same net. In a computer it corresponds to some procedure or algorithm which implements a particular method of data processing. On the other hand, each homogeneous net working as heteroassociative memory keeps associations between input and output images. This corresponds to rules of traditional programming, only blocks of those rules are kept in different, genetically predetermined hardware locations. You see that in a neurocomputer, the main paradigm is rule-based and types of rules are predetermined at hardware level.
So does a neurocomputer have a processor? Seemingly, yes, but it is striped of many functions known for its counterpart in a computer. Basically, if we regard the neocortex as the memory, the role of a processor will be to activate or suppress different areas and links between them. A good example is attention. That is, a processor of a neurocomputer accepts various signals from outside and inside the brain, but its output is just internal "turn on" or "turn off". Meanwhile there is something else. Software may be further subdivided into applications, system-level, and firmware (hardware emulation). While the first part is associated with the neocortex and constitutes various externally visible human abilities, the last is stored in a processor. Here we encounter an interesting turn. We have already seen that the memory of a neurocomputer is capable of processing. On the other hand, its processor is made of similar blocks of associative memory. Only now processing is regarded as a main function while memory provides storage for system programming.
Subcortical nuclei of the brain are well described both anatomically and functionally. Many of them provide low-level control which is unrelated to the psyche and behavior. Others are a part of motor and sensory systems. For example, the thalamus is a major relay station which conducts signals from sensory organs to the neocortex. It is the remainder that may be regarded as a candidate to the role of a processor. Two main parts may be named: the basal ganglia and the limbic system. Functional study of the basal ganglia hints that this is also memory indeed. Seeking analogy with computers, one can say that the basal ganglia are BIOS – the memory which contains the most basic subroutines. So the only part remains – the limbic system and its blocks are well connected into a functionally complete unit. Only not all of this complex may be regarded as a CPU. The limbic system is usually associated with motivation and emotions. Human motivation is subdivided into 2 absolutely different types which come from different parts of the brain. High-level goals and long-term plans are generated in the frontal neocortex, while biological desires such as hunger – in the hypothalamus. If we put aside such blocks, 2 major structures remain – the hippocampus and the amygdala.
Let's try and figure out how this processor operates in general.
A Turing machine is claimed to be universal. The human brain is even more universal. It can work in the completely analog mode when a person monitors some object and follows its movements. The first thing to do is to determine the class of tasks which the brain processor is used for. The brain as a whole is a regulator so if everything is normal, it may stay in the state of idle run. Despite not sleeping, it may do nothing special, just perform random actions without particular use. An alternative is goal-oriented behavior. That's what is interesting for us. That's when activity becomes highly structured and the processor operates at full power. The typical example is the task to reach some place walking in the city. Other tasks may be described by analogy. Consider manufacturing. The goal is to assemble some product from parts. Performed operations are separate steps or crossroads on the route, only motion happens in virtual space. The method to generate goal-oriented behavior is problem solving. That's the difference. Creativity. A Turing machine is designed to perform ready algorithms, the processor of the brain – to generate algorithms.