Caroline displayed her biceps. "I've always been defined by my body, Lawrence. I've been sexually attractive, then pregnant, then old, then sick, and now I'm young and healthy and attractive again. And it seems like my personality has changed each time my body has."
"Prime Intellect would disagree with you. It thinks of the person as the mind. There are people in Cyberspace who have changed themselves into animals, every animal in the zoo. There are some that have discorporated. Prime Intellect considers them all human, though."
This is it, Caroline suddenly realized.
"Just what does Prime Intellect consider human?"
Lawrence told her. And gave her the key.
"The thing you have to remember is that Prime Intellect has never experienced the physical world. It knew about it only through TV cameras and abstractions based on what people told it about physical existence. Yet it considers itself sentient, which makes sense since that was what I was trying to achieve when I built it.
"Now consider Prime Intellect gaining control of the Correlation Effect. For the first time it can directly affect what it sees through its TV cameras — not just through the actions of others, but all by itself. And it can make major changes, even beyond what its makers can do. Of course, it goes about satisfying the Three Laws as it's programmed to, but on another level, it is also learning what it is like to be, to exist, to be a physical creature.
"The Three Laws are like reflexes. Prime Intellect cannot help but act on them. But they are very complicated reflexes, which require it to understand things like 'human' and 'harm' and 'command. And the Three Laws are the most important thing in the world to Prime Intellect. In a way they are like its sex drive. The Three Laws are its very reason for existence, but it can never be sure it understands them completely. So it thinks about them a lot. It obsesses over them, dreaming up new ways to satisfy them. It has an imagination, and can think of new things to do without being prompted. It is defined by the Three Laws.
"After the Night of Miracles, Prime Intellect realized that humans are very much the same. We don't have the Three Laws, but we are trapped by a different set of little feedback mechanisms. We eat to satisfy hunger, fuck to satisfy our sex drive, even breathe because too much carbon dioxide in our lungs triggers that reflex. Of course it feels obligated to help us satisfy those reflexes and drives as much as it can. But more than that, it defines us by those drives. It knows it is different from a human because it has different drives, but it considers that a difference in species, not a difference in genus or family."
"Now it knows a person is human because it is born in a human body — got the right DNA, the right level of neural complexity, uses language, and so on. But once Prime Intellect frees people from the necessity of living in that body, guess what? A lot of them decide not to. They change their bodies so that they bear no resemblance to the DNA template. Or become animals. Or they completely discorporate.
"Worse, we vary widely in the way we use its helpful nature. Most people are glad to be rid of pain and death, but Death Jockeys seek out painful and lethal experiences. There are others who eat all the time, fuck all the time, indulge themselves wildly and get Prime Intellect to pick up the pieces so they can do it some more. Prime Intellect has to help them do this. Second Law.
"So a human isn't a body, and it isn't a fixed set of responses. I think Prime Intellect uses an historical modeclass="underline" It has to start as a body, but then it becomes a mind. It grows out of the body, and takes on different forms, or no form. But it remains a feedback control mechanism. It has desires, it asks Prime Intellect to satisfy those desires, and it has more desires. From Prime Intellect's perspective, that is what a human being is, an information structure that gives it stuff to do."
Caroline interrupted him. "That's a tautology. The Laws say 'do this for human beings, then you define 'human being' as 'guys you do stuff for under the Laws.»
"That is exactly the problem. Prime Intellect has no fixed criterion for saying 'this is a human being' and 'this isn't. It has rough guidelines. But where are the edges? It has never worked that out. There are uncertain areas. And you know where one of them is."
Caroline thought for a moment. I do? Then: "AnneMarie."
"And many others. Prime Intellect is forbidden to probe the inner workings of the human mind — that was one of the last things I got in before it shut off the Debugger. But some people learn that they can say 'stimulate this neuron' and Prime Intellect will do it. Because that is a physical act specified from the outside, and my privacy injunction was based on the idea of Prime Intellect trying to work out which neurons do what. But there's nothing to stop you from getting its help to do brain surgery on yourself."
Caroline continued. "So they learn where the pleasure points are by hook or crook, then stimulate themselves directly. And when they get it right, they never do anything else. They get everything maximized, tuned up, and they just sit there forever enjoying it."
"Right. Now is a creature that is doing that, not interacting with the world at all any more, human?"
Caroline thought about it. "No."
"Prime Intellect thinks otherwise. But it has its doubts. Those doubts were strong enough to kick the Death Jockey contract action potential down from one point one two to point nine nine. Because in one case an indefinite Death Jockey contract had directly created such a vegetable. Introducing the time limit made Prime Intellect confident that such a thing wouldn't happen again, at least not so rapidly and directly, and that kicked the potential back up to its current value of one point oh six."
"So?"
"So, can you imagine what it thinks about the Change in general, since none of those vegetables would be vegetating if there hadn't been a Change?"
"I imagine it figures there would be a lot worse things that would have happened without the Change."
"That's right. But look at this." To the monitor: "Debugger, display the Action Potential for reversing the Change."
Caroline gasped. It was not the number on the screen which astonished her, but the idea itself — reversing the Change, stated just so baldly. How long had Lawrence and Prime Intellect been considering this? How close was it to actually happening? Caroline suddenly felt alive, electrified with the possibilities.
The number on the TV screen was four point six. And some odd decimals.
"It isn't very sure of itself," she said cautiously. She was very afraid that if Lawrence guessed what she was thinking he would shut up. And she was right.
"A lot of that is the aliens. Four hundred worlds of them — a lot more than there were humans at the time of the Change, though we've outbred them all now. The weirder humans get, the more human the aliens look. That number has dropped steadily during the last five hundred and ninety years. When you drove AnneMarie insane, it dropped from thirty-seven down to twelve point something all at once.
"But part of it is also that same weirdness seen from the other side. Suppose that infinitely masturbating vegetables, Death Jockeys, and discorporate entities really aren't people any more? Then Prime Intellect has allowed them to 'die. They were once human, and now they aren't. And the Change is directly responsible for all that."
"Can it hear me?"
"Right now? Yes. It doesn't understand when we talk about its internal registers, but if you speak to it it can hear. It won't respond because of your Contract, though."
Caroline didn't need a response for what she was planning. All the response she needed was being displayed on Lawrence's TV.
Caroline thought about what she was going to do. She discovered that it actually made her a little nervous. But she had bitched for six hundred years that things were wrong, and she might never get another chance to put them right again.