Dad announced at the start of the drive that he would tell me everything about the Pioneer Project, but so far he hasn’t said a word. And to be honest, I’m not so anxious for him to begin. I’m in no rush to hear the details that horrified my mother. After her fit at Westchester Medical Center, she went home and locked herself in her bedroom, where she chose to stay rather than come with us to Colorado.
After a while the road curves sharply to the north, climbing higher into the mountains. The convoy slows to about thirty miles per hour and Dad shifts the SUV to a lower gear. He shakes his head. “Look at all the snow on the ground. Hard to believe it’s almost April. I guess spring comes late around here, huh?”
He glances sideways at me, clearly hoping for a response. But I’m not interested in talking about the weather, so I say nothing.
“I’d hate to get caught in a snowstorm on this road,” he persists. “Good thing it’s a sunny day.”
I feel sorry for him, actually. Dad’s terrible at communicating. He can’t talk about anything personal without getting upset, so he avoids difficult conversations. I was four when the doctors at Westchester Medical diagnosed my Duchenne muscular dystrophy, but it took Dad almost five years to work up the courage to explain what it meant for me. The same thing is happening now. He needs help getting started.
“We’re not here for the sightseeing, Dad. You said you were going to explain everything.”
He looks straight ahead, staring at the road. “I know, I know.” His Adam’s apple bobs up and down in his throat. “There’s so much to tell you. I don’t know where to begin.”
“Start with Sigma.”
He nods. His hands squeeze the steering wheel. “Sigma is the eighteenth letter in the Greek alphabet. In mathematical formulas it symbolizes the sum of a sequence of numbers.”
“Okay, I already knew that. What does—”
“We gave that name to our artificial-intelligence software because it was like a sum. Our research group developed Sigma by combining different kinds of AI programs.” He glances at me again, then turns away. “Some of the programs focused on pattern-recognition tasks, such as recognizing a face in the crowd. Other programs were more like the ones we designed for understanding language. They could find the answers to complex questions by searching through billions of documents and finding the connections.”
“You mean like QuizShow? The program that played Jeopardy!?”
“Yes, that was the prototype for a whole new class of AI software. Our strategy was to load all these artificial-intelligence programs into the neuromorphic circuitry we built and get them to compete with each other. Basically, we set up a computer version of Darwinian evolution. Only the strongest programs could survive.”
“Okay, you lost me. How can the programs compete with each other?”
“We tested each program to see how well it could imitate human reasoning and conversational skills. We deleted the less successful programs and allowed the more successful ones to advance to the next stage. Because the AI programs could learn from experience and rewrite their own software code, they started to redesign themselves to become better competitors. After six months, a clear winner emerged. That was Sigma, the first Singularity-level AI system.”
My chest tightens. Because I’m a computer geek I know what “Singularity-level” means. The Singularity is the hypothetical point in the future when machine intelligence will leap past human intelligence. Computer scientists have been predicting for years that machines will eventually become smarter than people. Now Dad’s saying this point is no longer in the future. It already happened.
“Wait a second. How smart is Sigma?”
“Impossible to say. The AI was already pretty intelligent when it won the competition a year ago. It had complete command of conversational English. And it had developed a sense of consciousness, an awareness that it was a thinking, intelligent entity.”
“How do you know that?”
“The program could gauge its own abilities. It asked questions about itself and its origins. It showed a strong desire to obtain more knowledge, and it developed strategies to satisfy this desire. Very clever strategies.” Dad grimaces. “And the AI has only become more intelligent since then. See, that’s the nature of a Singularity-level program. It’s always redesigning itself, so its capabilities grow very quickly.”
“And you let this program take over your computers?”
“No, we recognized the danger. We stopped all work on Sigma and locked the program in a secure server at the Unicorp lab, with no links to any other machines.”
“Well, it obviously figured a way out. Why didn’t you just erase it?”
Dad points at the Humvee that’s leading the convoy. “Peterson wouldn’t let me. The Department of Defense thought it could turn the program into a weapon. And they knew that researchers in other countries were doing similar work with artificial intelligence and neuromorphic circuitry.” He lowers his voice, even though no one can overhear us. “A few months after we developed Sigma, Peterson showed me a classified report about a project in China. A Singularity-level AI had apparently infected the computers at an engineering complex in Tianjin. The Chinese army had to destroy the building to prevent the AI from escaping.”
I shake my head, astounded. “And Peterson still wouldn’t let you erase Sigma?”
“The Defense Department knew it couldn’t stop all the AI projects around the world. There were too many of them, and some were in places like Russia and North Korea, where the U.S. military couldn’t go. We just had to accept the fact that sooner or later a Singularity-level program was going to escape from a lab somewhere. And the worst thing was, we had no idea what the consequences would be. The AI might pose no danger at all. It might harmlessly bounce around the Internet, observing everything but doing nothing. Or it might even be friendly. It might help us cure cancer or eradicate poverty.”
“Or it could be unfriendly,” I point out. “It might set off explosions and electrocute people.”
Dad bites his lip. “Yes, exactly. So Peterson gave me a new assignment. He asked me to predict what an AI like Sigma would do. And try to figure out how to make it friendly.”
“By rewriting the program, you mean? Writing ethical rules into its code? Don’t kill, don’t lie, that kind of thing?”
“That’s the first thing we tried on Sigma. But a Singularity-level AI has full control of its software code, so it can reverse any changes it doesn’t like. We tried all sorts of programming tricks, but nothing worked. Sigma is like a human that way. You can’t just cram something down its throat.”
“So what did you do?”
“I convinced Peterson to try something else. The Pioneer Project.”
I clench my good hand, steeling myself. “Why did you bring me here, Dad? And all the other kids too?”
He takes a deep breath. “Before I say anything else, I want to tell you how sorry I am. I just couldn’t accept what happened when you got sick. That’s why I started doing AI research. That’s why I’ve pursued it for the past ten years. I wasn’t doing it for Unicorp or the Army. I was doing it for you.”
I don’t understand. “Dad, what—”
“It has to do with the nature of the Singularity. It’s not just about machines becoming smarter than humans. It’s also about humans moving past their limitations. Becoming greater than their ordinary selves, transcending their ordinary lives. And as I thought about all the things that would become possible, I saw a way to save you.” Dad’s voice is weirdly high and choked. I think he’s trying not to cry. “The Pioneer Project turned my idea into reality. The Army gave me all the money and manpower I needed. But I didn’t have enough time to test it. I’m still not sure if it’ll work.”