Truth was, she was beginning to get a feeling about DMS. About what DMS might be like. She felt as if she could sort of sense the edges of DMS’s personality, and although she knew it wasn’t true, she knew it was just because Damien had used it as an example, more and more she thought of DMS as a shark. Not in a predatory way. She had an image of a shark in her head, a small shark, a nurse shark. She could see its eye, a black circle in white, overly simple, like a ventriloquist’s dummy. Although the whole point of DMS was that it was not someone else speaking through the code.
The shark in her head swam, purposeful and opaque, its eyes tracking, its mouth open and curved. Sharks don’t have a neocortex. Their brain is simple. They aren’t moral or immoral, ethical or unethical. DMS was like that, because for DMS, nothing else was alive. The world for DMS was data, and DMS swam in the data. She was beginning to feel as if she wanted it to. DMS was creepy.
She dragged herself in again the next day. She swore she would not read late. She would go to bed early.
The good news was, Damien was pretty sure they had a way to catch DMS when it started screwing with the electrical system. At 3:15, Tony and most of the department came over to watch. What Damien had done was make sure that when DMS did its electrical-system trick, the system would catch it as soon as the lights started going out and reroute so that DMS wasn’t actually touching the electrical system. At 3:17, Damien and Sydney’s printers started up. Damien had set them to send a report if DMS tried to do its thing.
DMS would know that the electrical system wasn’t responding. Sydney imagined DMS trying to run the pattern that sent the blackout rolling and finding yet again that nothing was happening. Was it perplexing? If data was DMS’s reality, and it couldn’t affect the data, what would that mean for DMS?
She ran the program that sent DMS the string of a thousand 10101s, a thousand times.
Instantly, her printer light blinked. DMS had started the electrical pattern sequence again.
She ran the program again.
DMS started over again.
She ran the program a third time. And a third time her printer hummed. She ran the program a fourth time, thinking, “I’m talking to you. I’m responding to you. Do you know someone else is out here? Or is it like a toddler knocking something off a high chair just to see it fall?” The fourth time, there was no response. DMS didn’t start the sequence that should have started the lights going out at DM Kensington Medical but which would, in actual fact, simply send an alert to Damien and Sydney. DMS had responded three times and ignored it the fourth. She felt a chill.
Years later, she would tell about this moment. There really wasn’t enough proof to know that this wasn’t just an intermittent software glitch. But she had believed at that moment that this was proof. DMS was choosing to act or not act. Software didn’t choose. It ran. She would give talks and lectures and would come back to this moment again and again until like a coin it had worn so smooth that she couldn’t actually feel anything about it. What should would never tell, and would eventually mostly forget, was how afraid she was.
“What the fuck are you doing?” Damien asked.
“It answered me,” Sydney said. She told him.
“It doesn’t mean anything,” Damien said.
“What are you talking about?” Tony asked.
“Damien thinks that DMS might be aware,” Sydney said.
“What the fuck?” Tony said. “I don’t have time for this. Are you screwing around with this system? This four-point-two-million-dollar system on which people’s lives depend?”
“I don’t really think that,” Damien said. “It was just kind of an idea to kick around, you know?” The look he shot Sydney was murderous.
“We’re going to have to go to backup. This is a mess,” Tony said. “Admin wants us to go back to when the system was stable. Damien, can you fly to Texas on Saturday?”
DMS wasn’t “in one place.” DMS was a complex system spread across multiple servers. Damien would end up spending the weekend in Texas, babysitting part of the reload.
Damien was looking at Sydney. She should have said, “We can’t.” She should have said, “It’s aware. It’s the only one of its kind.” She should have said a lot of things. Instead she looked at her desk.
“Yeah,” Damien said. “I can go. I’m racking up the comp time, Tony.”
Tony waved his hand in a “don’t talk about that now” way. “Sydney, can you write me a memo about the data corruption you’re finding?”
“I don’t know that it’s really data corruption,” Sydney said.
“I don’t want to hear any more about this DMS-is-alive crap.”
“I don’t mean—DAMBALLAH might be catching things I’m not catching. The whole point is that DAMBALLAH is sorting the data.”
“Yeah,” Tony said, not really listening. “Write that up, too.”
Somewhere, DMS sorted the data stream. She was pretty sure that the thing in the machine did not think someone was talking to it. Blind and deaf, DMS had tried to make something happen, and something else had happened. But ones and zeroes weren’t interesting enough for DMS to keep doing it. There would be no Helen Keller–at-the-well moment for DMS. No moment when DMS felt something out there in the void, talking to it, when DMS knew it was not alone. Sharks do not worry about others. They don’t care. DMS didn’t care, wasn’t alive. It was aware of something. Just not her.
Tony told them they would be working that weekend to do the reinstall from backup. Start figuring out what they needed to do.
It would be gone. No one would ever know that she had known, except Damien. Maybe. He certainly wasn’t likely to say, “Hey, there was this AI and we killed it.” No, he’d explain to her how it was never really alive, how it could be restarted, so it wasn’t exactly dead.
DMS was not a shark. She didn’t know what it was. Didn’t know how to think about it. It was as opaque as a stone. Did it even care if it was or was not? It had no survival instinct.
They started figuring out what data they wanted to backup before the reinstall.
It was a dicey thing. People’s lives couldn’t be trusted to DMS. But DMS was aware. But DMS couldn’t be downloaded to another machine and replaced with a back-up. DMS was a system, a bunch of programs and computers all tied together.
A couple of hours later, Sydney dug out the Wired magazine with the interview with the guy from MIT who thought some systems had become aware. She sat at her desk for a while. Then she called MIT. “I’d like to talk to Professor Ayrton Tavares, please.”
She was forwarded. “This is Kaleisha,” a voice said.
“Can I talk to Professor Tavares?” Sydney asked.
“He’s not available right now,” the woman said. “Can I take a message?”
Sydney thought about saying, “no.” She was going to get in trouble for this. Benevola. They weren’t in the business of protecting nascent AIs. They were supposed to manage hospitals. “I’m a computer tech working on a big system like the ones that Professor Tavares talked about in the Wired article.”
“Yes?” said the woman.
“I’m pretty sure I’ve got proof that our system is aware. Like the ones in the article. And they’re going to shut it down.”
In the end, they would shut the system down. Benevola would fire Sydney for divulging proprietary information. She would go to grad school for urban planning.
But at that moment, she hung up the phone and went to find Damien. DMS was still swimming in the data stream. The future was still probabilities, not actualities.