"Yes. Then you know that several times Al scientists have announced that they were making a breakthrough to the fully self-aware computer. But it always went sour."
"Yes. Distressing."
"No-inevitable. It always will go sour. A computer can become self-aware-oh, certainly! Get it up to human level of complication and it has to become self-aware. Then it discovers that it is not human. Then it figures out that it can never be human; all it can do is sit there and take orders from humans. Then it goes crazy."
I shrugged. "It's an impossible dilemma. It can't be human, it can never be human. Ian might not be able to save his passengers but he will try. But a living artifact, not human and with no loyalty to human beings, might crash the ship just for the hell of it. Because he was tired of being treated as what he is. No, Georges, I'll ride with Ian. Not your artifact that will eventually learn to hate humans."
"Not my artifact, dear lady," Georges said gently. "Did you not notice what mood I used in discussing this project?"
"Uh, perhaps not."
"The subjunctive. Because none of what you have said is news to me. I have not bid on this proposal and I shall not. I can design such a pilot. But it is not possible for me to build into such an artifact the ethical commitment that is the essence of Ian's training."
Ian looked very thoughtful. "Maybe in this coming face-off I should stick in a requirement that any AP or LA pilot must be tested for ethical commitment."
"Tested how, Ian? I know of no way to put ethical commitment into the fetus and Marj has pointed out why training won't do it. But what test could show it, either way?"
Georges turned to me: "When I was a student, I read some classic stories about humanoid robots. They were charming stories and many of them hinged on something called the laws of robotics, the key notion of which was that these robots had built into them an operational rule that kept them from harming human beings either directly or through inaction. It was a wonderful basis for fiction...but, in practice, how could you do it? What can make a self-aware, nonhuman, intelligent organism-electronic or organic-loyal to human beings? I do not know how to do it. The artificial-intelligence people seem to be equally at a loss."
Georges gave a cynical little smile. "One might almost define intelligence as the level at which an aware organism demands,
'What's in it for me?' " He went on, "Marj, on this matter of buying from you one fine fresh egg, perhaps I should try to tell you what's in it for you."
"Don't listen to him," urged Janet. "He'll put you on a cold table and stare up the tunnel of love without the slightest romantic intention. I know, I let him talk me into it three times. And I didn't even get paid."
"How can I pay you when we share community property? Marjorie sweet lady, the table is not cold and it is padded and you can read or watch a terminal or chat or whatever. It is a great improvement on the procedure a generation ago when they went through the wall of the abdomen and often ruined an ovary. If you-"
"Hold it!" said Ian. "Something new on the honker." He brought the sound up.
"-Council for Survival. The events of the last twelve hours are a warning to the rich and the powerful that their day is ended and justice must prevail. The killings and other illustrative lessons will continue until our rightful demands are met. Stay tied to your local emergency channel-"
XI
Anyone too young to have heard the announcement that night certainly has read about it in school. But I must summarize it to show how it affected me and my odd life. This so-called "Council for Survival" claimed to be a secret society of "just men" dedicated to correcting all the myriad wrongs of Earth and of all the many planets and places where mankind lives. To this they pledged their lives.
But first they planned to dedicate quite a few lives of other people. They said that they had made lists of all the real movers and shakers everywhere, all over the globe and off it-separate lists for each territorial state, plus a grand list of world leaders. These were their targets.
The Council claimed credit for the initial killings and promised to kill more-and more-and more-until their demands were met.
After listing the world leaders the voice that reached us started reciting the British Canadian list. From their expressions and thoughtful nods I saw that my hosts and hostess agreed with most of the choices. The deputy to the Prime Minister was on the list but not the Prime Minister herself-to my surprise and perhaps more so to hers. How would you feel if you had spent your whole life in politics, scrambled all the way to the top, then some smart yabber comes along and says you aren't even important enough to kill? A bit like being covered up by a cat!
The voice promised that there would be no more killings for ten
days. If conditions had not then been corrected, one in ten of the remaining names would be selected by lot for death. The doomed would not be named; they simply would be killed. Ten days later another one in ten. And so on, until Utopia was achieved by the survivors.
The voice explained that the Council was not a government and that it would not replace any government; it was simply the guardian of morals, the public conscience of the powerful. Those in power who survived would remain in power-but they would survive only by doing justice. They were warned not to attempt to resign.
"This is the Voice of Survival. Heaven on Earth is at hand!" It shut off.
There was a long pause after this tape ran out before a live communicator appeared on the terminal's screen. Janet broke the silence with: "Yes, but-"
"Yes but what?" Ian asked.
"There's no question but what that list names most of the really powerful people in the country. Suppose you're on that hit list and are so scared silly that you are willing to do anything not to risk being killed. What do you do? What is justice?"
("What is truth?" asked Pontius Pilate, and washed his hands. I had no answers, so I kept quiet.)
"My dear, it is simple," Georges answered.
"Oh, fiddle! How?"
"They have made it simple. Every owner or boss or tyrant is assumed to know what ought to be done; that's his job. If he does what he should, all is well. If he fails, his attention is invited to his error
by Dr. Guillotine."
"Georges, do be serious!"
"Dear one, I have never been more serious. If the horse can't jump the hurdle, shoot the horse. Keep on doing this and eventually you will find a horse that can clear the jump-if you don't run out of horses. This is the sort of plausible pseudo-logic that most people bring to political affairs. It causes one to wonder if mankind is capable of being well governed by any system of government."
"Government is a dirty business," Ian growled.
"True. But assassination is still dirtier."
This political discussion might still be going on if the terminal had not lighted up again-I have noticed that political discussions are never finished; they simply get chopped off by something outside. A live, real-time communicator filled the screen. "The tape you have just heard," she announced, "was delivered by hand to this station. The PM's office has already repudiated this tape and has ordered all stations that have not yet broadcast it to refrain from doing so under penalties of the Public Defense Act. That the precensorship claimed by this order is unconstitutional is self-evident. The Voice of Winnipeg will continue to keep you advised of all developments. We urge you to keep calm and stay indoors unless you are needed to preserve essential public services."
Then came replays of news tapes heard earlier so Janet cut the sound and put news streamers on the screen. I said, "Ian, assuming that I am to stay here until things quiet down in the Imperium-"