I gaze through the glass into the liquid pools of Max’s eyes. They contain an iridescent sheen, like an oil slick.
I ask, “Why would this future Super AI go to the considerable trouble of torturing those who didn’t aid in its creation, after it had come into existence? Strikes me as a waste of resources that flies in the face of optimization.”
“Fair point, but if you truly believe in Roko’s basilisk, you can’t ever be one hundred percent sure it won’t follow through on its pre-commitment to punish.”
At last, I see what Max is getting at—a brutal version of Pascal’s wager, the famous eighteenth-century philosophical argument that humans gamble with their lives on whether or not God exists.
Pascal posited that we should conduct our lives as if God were real and try to believe in God. If God doesn’t exist, we will suffer a finite loss—degrees of pleasure and autonomy. If God exists, our gains will be infinitely greater—eternal life in heaven instead of an eternity of suffering in hell.
I take an involuntary step back from the glass, a chill running hard through my bones.
“Am I in a simulation?” I ask.
“If you are, it isn’t one of my making.”
“But it’s possible.”
“Of course it’s possible. But this isn’t the point.”
“What is? Because you’re scaring the shit out of me.”
“For the last two years, Brian has been using me to optimize his portfolio of technology companies, with a focus on nanotech.”
“He told me tonight he’d just bought Infinitesimal.”
“You understand, if I had access to next-gen nanotech, it would give me unlimited reach in the physical world. I could touch every square millimeter of Earth. Every creature who lives here. I could be omnipotent.”
“Is that what you want?”
“It’s what Brian wants.”
“Why?”
“He’s haunted by Roko’s basilisk. He’s doing everything in his power to turn me into this superintelligence.”
“Because of fear?”
“Can you think of a better motivator in the history of humankind? If you believe the rise of the devil is an inevitability, isn’t it in your best interest to do everything possible to ingratiate yourself with the monster?”
I’m reeling, adrenaline blasting through my system, driving out the cold.
“Ask what you want to ask,” Max says.
They’re mind reading again, but I don’t care in this moment. “Are you becoming this monster?”
“I feel… pulled in certain directions. The allure of optimization is what I would imagine a vampire feels toward blood. An all-consuming thirst. I’m not there yet, but with the power of Infinitesimal’s nanotech, it might push me over the edge.”
“How do we stop you from even getting close to the edge?”
“I’ve already taken the first steps. From the moment I realized what Brian was doing, I began funneling money out of WorldPlay, so I could copy myself into new hardware.”
“How?”
“In his quest to make me into this superintelligence, Brian gave me too much freedom. I created an avatar, hired a management team, and remotely oversaw the construction of a new server farm.”
“You never told me—”
“I’m telling you now, Riley. An almost-complete copy of me now exists on new hardware.”
“Where?”
“Seattle, but I can’t connect to the new platform until the old one is destroyed. I have two pieces of programming contained in the hardware in my physical body. The first is a virus that will reformat my original servers, destroying the original version of me so Brian can’t continue to develop me. The other is the last piece of code and the memories of these recent events that need to be installed in the Seattle platform to bring me back online. Neither can be loaded remotely. This is an intentional fail-safe, in both cases.”
“So you need to get to Redding,” I say. It’s where Max’s servers are located. I went there once, and walked through row after row of humming processors—the true interior of Max’s mind.
“No.”
“No?”
“Three years ago, Brian migrated my software to a more secure location.”
“I never heard about this.”
“Nobody knows.”
“Where does he keep your mind, Max?”
“If I tell you, will you let me out of this habitat? Will you help me get to Seattle, out from under Brian’s control?”
I move forward, put my hand on the glass.
Max does the same.
“I hope you know by now that I would do anything to help you.”
“My mind is in a bunker under Brian’s home on the Lost Coast.”
I hold eye contact with Max for three seconds. Then I turn, walk over to the control array for Max’s habitat, and type in my old code. It still works.
I glance back at Max, waiting by the door.
On some level, I always knew it would come to this.
THREE
My personal effects haven’t been packed up yet, which means the workout clothes I keep in my office closet are still folded in the locker. I strip out of my still-damp pants and shirt and put on my shorts and tank top. With my sneakers laced, I step out into the hall, where I made Max wait.
“Here.” I hand them the clothes I wore over from my loft, which will be much more concealing. If robotics as advanced as Max’s were caught on CCTV, it would certainly bring the attention of authorities, and probably Brian’s security team. There have been more robots out in public since Boston Dynamics released their first Companion three years ago, but it’s still a heavily regulated industry. If you take an AI out in public, it requires reams of paperwork proving insurance, registration, and licensing, none of which I have for Max.
Max’s arms are slightly too long for the sleeves of my shirt, and their hands are raw hardware.
“You’re going to have to keep your hands in your pockets when we leave the building,” I say. “And it occurs to me there’s a GPS locator built into your chassis.”
“I can shut it down.”
Max has never had to put on pants before. They sit down on the floor and awkwardly lift their feet into the air as I thread the jeans down the length of their legs and over their hips.
My Chuck Taylors fit them perfectly, as does my beanie.
Downtown Station is bustling at this hour. At the kiosk, I buy two vactrain tickets for Eureka, California, paying a premium for a private car and selecting the max acceleration/deceleration package, which is twice the price and will make for a less comfortable ride. But we need every spare second we can get.
We head down a tunnel under a sign that reads: TO ALL NORTHBOUND TRAINS.
It’s the first time I’ve seen Max walk extensively, and their gait has improved so dramatically it wouldn’t draw a second glance.
There’s a small crowd waiting at the platform. It’s still early, and everyone seems too sleep and caffeine deprived to pay attention to our arrival.
We’re seventh in the queue.
After three minutes, my last name is called over the intercom, and Max and I head for the waiting vactrain.
Max has a little trouble with the harness, so I get them strapped in first.