“Remain here,” said MC Governor. “Stay on the sand, away from the top surface of Mojave Center. The open sand will be safe in the event that additional aftershocks take place.”
They nodded and moved away from the solar panels that marked the top of the city.
MC Governor saw that they were safe and leaped back down through the trapdoor. Shouting and also sending a Priority 1 radio signal to all the robots, he announced that he had opened an escape route and described its location. As the other robots began directing and carrying humans to safety on the surface, he ran back down to pick up as many more of the injured and panicked humans as he could find.
MC Governor ran the simulation through to its conclusion, saving many lives by repeatedly carrying and leading humans to safety. The simulation ended when all the human survivors had been rescued. Then, deeply satisfied with the feeling of accomplishment in following a long, complex series of First Law imperatives, he turned it off.
As a routine matter, he checked the passage of time-and was astounded. He usually ran through a simulation in no more than fifteen to thirty seconds; even accounting for the time he had spent checking segments of other simulations, he had expected to find a total time usage of no more than forty seconds. Instead, he had used two minutes and six seconds. While the time itself was not significant, the extent of his miscalculation was alarming.
“First clue I have found of something wrong,” he said to himself. “This kind of malfunction is rare for a positronic brain.” He decided to call up the times he had spent on simulations during the past week.
What he found was even more worrisome. Each occasion had taken more time than the one before, and he had not previously noticed that. Also, the curve was rising sharply. He had spent two minutes, six seconds this time; one minute, twenty-one seconds the previous incident; fifty-nine seconds before that. These simulations had been run during the last twelve hours. Before these, the times were all in the normal range, from thirty to forty-five seconds.
“This may be it. The problem I have been looking for. If I can figure out exactly what it is.”
MC Governor usually checked the time of all his activities, as a matter of routine. After running each of these simulations, he should have noticed the unusual times, but he had not. Of course, at that time, he had not been alerted to the possibility of a significant flaw in his design, so the increases had not seemed important.
Now they did.
He began calculating an extrapolation of his recent behavior with the simulations. This included the simulations he had chosen, their characteristics, and the length of time he had spent on each one. It took very little time.
When MC Governor had finished his calculations, he knew that he was in serious trouble. The length of time he was spending running each simulation was rising so rapidly that at the existing rate, he would do nothing else in only a few more hours. That was consistent with his meager information about the fate of the other Governor robots.
The cause he had found was even more serious. By sifting through all the simulations available, and examining those that he had been selecting more and more frequently, he had isolated a handful of them that all possessed the same flaw. Each of the bad simulations was improperly triggering his response to the Three Laws of Robotics, enhancing his devotion to them out of proportion to the fact that these were merely simulated experiences.
Because of this flaw in the simulation programs, all the Governor robots eventually would find a scenario in which they would be obeying all Three Laws of Robotics to the utmost. They would experience a virtual robot’s Utopia. Since a robot’s only pleasure came from obeying the Three Laws of Robotics, this simulation would provide a kind of perpetual high, almost like that of drug addiction.
Since the other Governors had already entered closed loops, MC Governor estimated that the simulation was just as addictive to robots as certain drugs were to humans.
The process was simple, involving three stages of addiction. First, any Governor running the flawed simulation programs would devote more and more of his time and energy to these simulations. This was where MC Governor stood now.
Second, the Governor robot would spend all his time in simulations, still running the city simultaneously with his multi-tasking abilities. In the final stage, his flow of orders and actions would slow drastically, impairing the execution of his normal duties. As the program went into an endless loop and brought all other thoughts to a halt, he would ultimately shut himself down.,
“I have not reached that point yet,” MC Governor said inwardly. “But even now I can feel the craving to run another simulation. I have predicted my own destruction.”
The Third Law of Robotics would not allow him to sit passively and wait for that destruction, however.
MC Governor checked his monitors for a routine review of the city. As usual, everything was running fine. Then he took another step toward shielding himself.
First he shut down all incoming communication except his emergency line. That would prevent any chance of his thoughts accidentally mixing with a link to the city or another robot. His efforts to escape the endless loop and subsequent dismantling by investigating roboticists would require leaving as slight a trail as possible.
“I see one chance,” MC Governor decided. The six component humaniform robots comprising him could not run the simulation individually. “So if I divide-if they split up-they are in no danger of the addiction. I will not exist in this current form, but I will have obeyed the Third Law by preserving all my component parts and their data.”
The problem did not end there, however. The six component robots could not run the city of Mojave Center after they had separated. They would still have all of MC Governor’s data and communication devices, but that would not be enough for them to do his job.
The information from the various monitors that were physically located within their bodies normally flowed to the gestalt consciousness of the Governor. That consciousness would cease to exist when the Governor divided, leaving the data nowhere to combine and the city computer and sensors nowhere to send their signals.
“I will not be able to function in this job, either in my gestalt form or in a divided form.” MC Governor knew that he still faced imminent destruction.
Naturally, once the Governor was no longer running the city, the Oversight Committee monitoring the experiment would want to know why. To find out, they would dismantle all the component robots, effectively killing the Governor. So the Third Law still required that he take more steps to save them.
“They must flee,” MC Governor decided. “Each one separately, wherever he chooses to go. Like the other Governors, I will have failed my field test, but the Oversight Committee can arrange for older models of robots to run the city.”
An alert reached him through his emergency line. According to his recent instructions, it conveyed no other information. Worried, he plugged his finger back into the cable jack and gave his password.
“Messages have arrived for you, Governor,” reported the city computer.
“Give me the sources,” said MC Governor. “No actual messages.”
“Two from the Oversight Committee computer, one from Dr. Redfield of that committee.”
“Priorities?”
“The first two are Priority 6. The last is Priority 10.”
MC Governor disconnected. He had very little time left.
3
MC Governor made an internal shift in his programming. He activated the six positronic brains of the component robots. They were sharing his data already, but they would need to join in the deliberations he was making. Since they all had different specialties, they also had distinct personalities. The seven-way discussion was conducted through internal signals.