“I’m glad somebody can show initiative,” Bithras said.
I was as curious as anybody to find out what Alice and Jill would discuss.
Jill was the oldest thinking being on Earth, a fabulous figure, the first thinker to achieve bona fide self-awareness, as defined by the Atkins test.
Decades before Jill and Roger Atkins, Alan Turing had proposed the Turing test for equality between human and machine: if in a conversation limited to written communication, where the human could not directly view the correspondents, a person could not tell the difference between a machine and another human, then the machine was itself as intelligent as a human. This subtle and ingenious test neglected to take into account the limits of most humans, however; by the beginning of the twenty-first century, many computers, especially the class of neural net machines becoming known as “thinkers,“ were fooling a great many humans, even experts, in such conversations. Only one expert consistently pierced the veil to see the limited machines behind: Roger Atkins of Stanford University .
Jill outlived Atkins, and became the model for all thinkers built after. Now, even an exported thinker such as Alice could outstrip Jill several times over, but for one crucial quality. Jill had acquired much of her knowledge through experience. She was one hundred and twenty-eight years old.
We paid for the broadband connection between Alice and Jill, agreed to the encryption algorithm, and went to bed.
Sleep on Earth, despite my bichemistry, almost invariably felt heavy. The strain of Earth’s pull on a Martian’s muscles and organs could not be eliminated; it could only be treated. While I felt well enough awake, my sleeping self often drowned, dragged under shallow waters rushing in tides past fantastic, ivory-colored castles on ruby-colored islands.
I climbed or rather glided up the internal spiral of a tower staircase when Bithras shook me rudely awake. I reflexively jerked the covers up, fearing the worst. He pulled his hands back, eyes wide, as if deeply hurt. “No nonsense, Casseia,” he said. “There is a serious problem. Alice woke me. She’s finished her conversation with Jill.”
Allen, Bithras and I sat in our robes in the living room, cradling cups of hot tea. Alice ’s image perched primly on the couch between Bithras and Allen, hands folded on her knees. She spoke with a calm, deliberate voice, describing her encounter with Jill. Allen quietly made notes on his slate.
“The meeting was extraordinary,” Alice began. “Jill allowed me to become her for a time, and to store essential aspects of her experiences in my own memories. I provided her in turn with my own experiences. We divided our five minutes between conversation in deep-level thinker language, transfer of experiences, and cross-diagnostic, to see whether bad syncline searches could occur in any of our neural systems.“
“You allowed Jill to analyze your systems?” Allen asked with some alarm, looking up from his slate.
“Yes.”
“Tell them what she found,” Bithras said.
“This is in a sense proprietary,” Alice said. “Jill could face difficulties if her work is discovered.”
“You have our promise of discretion,” Bithras said. “Casseia? Allen?”
We swore secrecy.
“Jill considers all thinkers to be part of her family. She feels responsible for us, like a mother. When thinkers converse with her, she analyzes us, adding to her own store of knowledge and experience, and determines whether we are functioning properly.”
I detected reticence. Alice did not want to get to the point.
“Tell us, Alice ,” Bithras encouraged.
“I still feel deeply embarrassed by what Jill discovered in me. I am able to fulfill my duties, I am sure, but there may be reason to no longer trust my ultimate performance — ”
Bithras shook his head impatiently. “Jill found evolvons,” he said.
“In Alice ?” Allen asked, lowering his slate.
I sucked in my breath. “What kind?” I asked.
Alice ’s image froze, flickered, and went out. Her voice remained. “I am changing modes of display to better conform with my internal state,” she said. “I will not maintain a cosmetic front. Evolvons exist in my personality configuration. They appear to be original, not implanted after my incept date.”
An evolvon could be nearly any thing or system designed to exist in time, consume energy or memory, and reproduce itself. All living things were evolvons in a sense. Within computers and thinkers, the word usually referred to algorithms or routines not known to be part of the status design or acquired neural configuration — sophisticated viruses.
“Do you know their purpose?” I asked.
“Jill discovered them only by comparing my full configuration with my neural bauplan, my self-known design, and running a trace of her own devising. There are parts of me that are not known to me, and which I have no control over; these parts are not functional in my personality configuration. They have no known utility, but all of them contain reproductive algorithms. They are well-hidden. No traces on Mars revealed their presence.”
“Evolvons,” Allen said, his face pale. “That’s against the law.”
“I have difficulty describing my sensation at making this discovery,” Alice said. I wanted to hold her, but of course she had nothing to hold. Her voice remained level — I had never heard a thinker express negative emotions in speech. But her tone became a shade harsher as she said, “I feel violated.”
“Is it possible the evolvons have been planted since we left Mars or arrived on Earth?” Bithras asked.
“Very unlikely. I have not been accessed by specialists for repair, which would be the only way they could be planted after my incept date.”
Bithras folded his hands on his knee. “If you have these… evolvons, then Alice One has them as well.”
“Most likely,” Alice said.
“They were copied from her to you. And they escaped our most expert traces. That means they were planted by the manufacturer, right here on Earth.” “ The implications were jolting.
“I apologize for my inability to be trustworthy,” Alice said.
“No need to apologize,” Bithras said. “We’ll remove the evolvons — ”
“Jill does not believe that can be done without great care to avoid damaging my personality. They are imbedded in key routines.”
“Do you know what will activate them?” I asked.
“No,” Alice said.
“Can you guess?” I pursued.
“Specific triggering codes delivered by any of my inputs,” Alice said.
“They are sabotage,” Bithras observed, “waiting to happen.”
“Who’s responsible?” I asked.
“Earth,” he said, lips curling. “Sane, wonderful Earth.”
Bithras sent an emergency message to Mars, contents unknown to us, and returned to his bed, exhausted, soon after. Allen and I stayed up, ordered a bottle of wine, and sat drinking, talking with Alice .
“The most important thing,” I said, finishing the first glass, “is whether Alice wants to continue working with us.”
“Bithras and I have discussed this,” Alice said.
Allen and I felt tired and sad and discouraged, as if suffering through an illness in the family. What was dying rapidly was any joy we might have had, coming to Earth; any feeling of value as representatives of Mars, any sense of self-worth whatsoever. We were isolated, our friend was compromised in such a way that we could no longer have faith in her…
“What did Bithras say?” I asked softly.
“He believes I should carry on with my duties. I will of course be glad to continue.”
“Can you tell… ?” Allen asked, not finishing.
“I will not know when or if an evolvon is activated. This I have told Bithras.”
“Everything we set out to do is being scuttled,” Allen said, twirling his glass in his hand. “We can’t trust anybody or anything here.”