Ling forced herself to smile, forced herself to turn back to the human, the way her mother had taught her, forced herself to say the insipid words.
“May I please have a few minutes more, teacher? I like to watch the rain.”
“Well.” The tutor sounded surprised. “Since you asked so politely, you may.”
Chen surrendered his electronics at the next checkpoint. Then the security man slowly and thoroughly wanded him, looking not for weapons, but for any device that could possibly carry data in or out of the PICC.
The guard finally declared him clean, and Chen stepped forward and into the cavernous elevator. The doors closed behind him, and the elevator started its descent through one thousand meters of bedrock and towards the mad software entity that was all that remained of his dead wife.
Ling sifted through exabytes of data. Cryptographic libraries. High resolution satellite imagery. Whole brain scans. Genome sequences. None of it was her mother.
She looked for maps, physical maps, network maps. She found them. The network topology told her little. Nothing obviously fit the description of the quantum cluster her mother existed in. The physical blueprints of the building were no more helpful. Multiple data centers existed here, but their functions weren’t clear.
Ling kept searching. She would find her mother here.
The room-sized elevator took Chen down through the rock beneath Shanghai. A lit sign declared that the current PICC status was
ISOLATION IN EFFECT
All this was a precaution. Computer scientists, philosophers, futurists, writers of speculative fiction – they’d all written about the dangers of runaway superintelligence. If humanity ever created a being of radically increased mental capabilities, it placed itself at grave risk. That new being could be benevolent, of course. That would be the hope. Or it could be malicious, or simply indifferent to humans. It could seek to change the world in ways that it saw as improvements, but which were incompatible with the interests of its creators.
A superintelligent being might also be able to improve on itself, reaching into its own structure and finding ways to optimize them, to make itself smarter than its creators could have, with no obvious end in sight.
And for that reason, Su-Yong’s ability to edit herself was limited to superficial layers only.
Chen himself doubted the risk of runaway self-improvement. Intelligence showed diminishing returns. Just as a single human could not design a human-level intelligence from scratch, no superhuman creature could possibly design a creature of its own intelligence or greater. Oh, it might be able to make improvements on the methods used by its creators, and get some boost, but without collaborators, without access to new hardware, the improvements would level off.
And so he’d stretched the rules, hidden a few upgrades of his wife’s design in with more prosaic maintenance, made the case for upgraded quantum cores. All for sensible reasons, of course. All so she could produce more work of value to the Science Ministry and the State and humanity. All for the greater good.
Regardless, in this world, this world where everything was linked, where data ruled all, where cryptographic codes had replaced physical locks on the world’s wealth, on its infrastructure, on its weapons… In this world a being able to process information more rapidly than humans was the ultimate threat.
It was for such reasons that the Copenhagen Accords prohibited any attempt to create a non-human self-aware being. And for the same reasons, Chen’s own government, in sponsoring the creation of exactly such a dangerous and illegal being, had taken extreme precautions that it could be physically isolated, cut off from the outside world, and destroyed remotely if necessary.
The elevator clanged to a halt. Here, at the bottom, where no wireless transmission reached, three physical data lines connected to the outside world. One linked the Quantum Cluster to the net. That cable was physically disconnected now, its ends separated by a gap of ten meters.
The second cable carried data one way, from a grid of cameras and other sensors, up to the Secure Computing Center above. It let the SCC observe what happened here.
The third cable carried far simpler data. It connected a terminal above to the nuclear battery that provided power for the PICC. If things ever went ultimately wrong, that cable would carry a single command, instructing that nuclear battery to go critical in a chain reaction that would melt the underground facility to slag.
The wall-sized doors to the elevator parted. The meters-thick inner blast doors parted a moment later, and Chen Pang strode out to inspect his wife.
Ling frowned. There was no evidence of her mother here. But she knew that her mother was in the quantum cluster beneath Jiao Tong. And Father had gone there.
“Ling, your break is over now.”
Ling ignored the tutor. Where was her mother? Where?
Chen sat at the terminals that monitored his dead wife’s quantum brain and initiated the systems check. Through the bulletproof glass he could see the grid of liquid helium pressure vessels, the vacuum chambers a thousand times colder than interstellar space within them, containing the quantum processors in an environment almost completely devoid of thermal noise. He could see directly into the brain of this creature that he’d once been married to.
Data scrolled across the screens within seconds. The level 0 diagnostics were clean. Pressure vessels intact. Quantum bandwidth across the interconnects was excellent. Qubit coherence was well within the limits of quantum error correction.
The level 1 diagnostics came back next. Processor and memory utilization were high. She was furiously thinking in there. Requests for external data connections were nearly continuous. Millions of times per second she was trying to reach the outside net, the cameras, the audio pickups, the Nexus-band radios, the long-range link to the clone that had died in Thailand.
The level 2 diagnostics were the most disturbing. Her simulated brain looked less and less healthy. Her virtual brainwaves were chaotic and incoherent, inhuman looking. Neuronal interconnectivity in her frontal lobes looked terrible. The remaining virtual neurons there were working at a frenetic pace, trying to make up for the deficit.
It was true, then. She was going mad. And he had been rendered powerless to stop it.
Give me just one more insight, wife. This last breakthrough. Then you can die.
Chen Pang reached up and physically turned on the cameras and microphones that connected this room to his dead wife’s mind.
“Ling!”
Something was wrong, she realized. Father’s phone and slate had stopped moving. She thought he’d simply stopped somewhere, but when she interrogated them, they were out of contact with him.
“Ling, are you listening to me?”
She looked through the security cameras inside the center. Where was Father? Not in the hallways. Not in the main work areas. Not in the data centers. Not in the physical electronics labs. Where?
“Ling!” The tutor grabbed her arm, and Ling struggled to pull it free.
Wait. There. Not Father. But his phone and slate. They were on a table, behind a security guard. A checkpoint. An elevator door beyond that. There was another level!
She went back to the network topology, to the physical blueprints. There. Data lines that extended down. Repeaters on them, indicating that they went far. A network connection. She reached out for it.
Input burned itself into Su-Yong Shu’s mind.
Video.
Audio.
Real-time.
Here.
Her husband, Chen. He was here. He hadn’t abandoned her! Hope blossomed in Shu. She struggled to get a grip on herself, exerted a superhuman effort at coherence, at communicating what she needed.