Two seminar talks I gave shortly after my return turned out to have good or lucky ideas and led to successful further developments. One was on what was later called the Monte Carlo method, and the other was about some new possible methods of hydrodynamical calculations. Both talks laid the groundwork for very substantial activity in the applications of probability theory and in the mechanics of continua.
The hydrodynamical calculations were for problems in which there was no hope for closed formulae or explicit classical analysis solutions. They could be described as a sort of "brute force" calculations using fictitious "particles" that were really not the fluid elements but abstract points. Instead of considering individual material points of the fluid, it was a matter of' using the coefficients of an infinite series into which the continuum was developed as abstract particles for a global description of the fluid. The whole motion is described by some infinite series whose terms are successively less important. Considering only the first few of them, one changed the partial differential equations of several variables (or the integral equations in several variables) into ordinary or totally different equations for a finite number of abstract "particles.'' Some years later, the work of Francis Harlow in Los Alamos deepened, enlarged, and multiplied the scope of this approach to the calculations of motions of fluids or of compressible gases. These are now widely used. The possibilities of such methods have not yet been exhausted; they could play a great role in the calculations of air movements, weather prediction, astrophysical problems, problems of plasma physics, and others.
The second talk was on probabilistic calculations for a class of physical problems. The idea for what was later called the Monte Carlo method occurred to me when I was playing solitaire during my illness. I noticed that it may be much more practical to get an idea of the probability of the successful outcome of a solitaire game (like Canfield or some other where the skill of the player is not important) by laying down the cards, or experimenting with the process and merely noticing what proportion comes out successfully, rather than to try to compute all the combinatorial possibilities which are an exponentially increasing number so great that, except in very elementary cases, there is no way to estimate it. This is intellectually surprising, and if not exactly humiliating, it gives one a feeling of modesty about the limits of rational or traditional thinking. In a sufficiently complicated problem, actual sampling is better than an examination of all the chains of possibilities.
It occurred to me then that this could be equally true of all processes involving branching of events, as in the production and further multiplication of neutrons in some kind of material containing uranium or other fissile elements. At each stage of the process, there are many possibilities determining the fate of the neutron. It can scatter at one angle, change its velocity, be absorbed, or produce more neutrons by a fission of the target nucleus, and so on. The elementary probabilities for each of these possibilities are individually known, to some extent, from the knowledge of the cross sections. But the problem is to know what a succession and branching of perhaps hundreds of thousands or millions will do. One can write differential equations or integral differential equations for the "expected values," but to solve them or even to get an approximative idea of the properties of the solution, is an entirely different matter.
The idea was to try out thousands of such possibilities and, at each stage, to select by chance, by means of a "random number" with suitable probability, the fate or kind of event, to follow it in a line, so to speak, instead of considering all branches. After examining the possible histories of only a few thousand, one will have a good sample and an approximate answer to the problem. All one needed was to have the means of producing such sample histories. It so happened that computing machines were coming into existence, and here was something suitable for machine calculation.
Computing machines came about through the confluence of scientific and technological developments. On one side was the work in mathematical logic, in the foundations of mathematics, in the detailed study of formal systems, in which von Neumann played such an important role; on the other was the rapid progress of technological discoveries in electronics which made it possible to construct electronic computers. They, in turn, provided such a quantitative increase in the speed of operation so much greater than the mechanical relay machines that it produced a qualitative change and vastly improved and enlarged the use of the tool. The results are now known to everyone: computers introduced a new age in heuristic research, in communication, and in making the space age possible.
The number of applications in exact science, in the natural sciences, and in everyday life is so great that one can talk of "the age of computers and automata" as having begun.
At that time the computers were merely in statu nascendi. As a joke I proposed to make Monte Carlo calculations by hiring several hundred Chinese from Taiwan, gather them on a boat, have each one sit with an abacus, or even just pencil and paper, and make them produce the random numbers by some actual physical process like throwing dice. Then someone would collect the results, and total the statistics into single answers.
Von Neumann played a leading role in the launching of electronic computers. His unique combination of gifts, his interests, and traits of character suited him for that role. I am thinking of his ability, and inclination to go through all the tedious details of program planning, of executing the minutiae of putting very large problems in a form treatable by a computer. It was his feeling for and knowledge of the details of mathematical logic systems and the theoretical structure of formal systems that enabled him to conceive of flexible programming. This was his great achievement. By suitable flow diagramming and programming, an enormous variety of problems became calculable on one machine with all connections fixed. Before his invention one had to pull out wires and reconnect plug boards each time a problem was changed.
The Monte Carlo method came into concrete form with its attendant rudiments of a theory after I proposed the possibilities of such probabilistic schemes to Johnny in 1946 during one of our conversations. It was an especially long discussion in a government car while we were driving from Los Alamos to Lamy. We talked throughout the trip, and I remember to this day what I said at various turns in the road or near certain rocks. (I mention this because it illustrates what may be multiple storing in the memory in the brain, just as one often remembers the place on the page where certain passages have been read, whether it is on the left- or right-hand page, up or down, and so on.) After this conversation we developed together the mathematics of the method. It seems to me that the name Monte Carlo contributed very much to the popularization of this procedure. It was named Monte Carlo because of the element of chance, the production of random numbers with which to play the suitable games.
Johnny saw at once its great scope even though in the first hour of our discussion he evinced a certain skepticism. But when I became more persuasive, quoting statistical estimates of how many computations were needed to obtain rough results with this or that probability, he agreed, eventually becoming quite inventive in finding marvelous technical tricks to facilitate or speed up these techniques.
The one thing about Monte Carlo is that it never gives an exact answer; rather its conclusions indicate that the answer is so and so, within such and such an error, with such and such probability — that is, with probability differing from one by such and such a small amount. In other words, it provides an estimate of the value of the numbers sought in a given problem.