“All of this evidence suggests to me that even if we don’t understand ELOPe, it has already, in some sense, figured out that it is in a symbiotic relationship with humans, and that the best way to ensure its own success is to ensure our success. Our success as a team, our success as a company, our success as the human species. David and Gene, you guys want to throw this all away, simply because you don’t understand it and don’t trust it. Even if that’s true, I think there’s a very strong chance that ELOPe is a good thing for humanity.”
“Enough already!” David banged his fist on the table, and then jumped back to his feet. “Are you forgetting that ELOPe told you your father had a heart attack? That we have every reason to believe that it killed Bill Larry when he flew out to visit one of the offshore data centers? How are those good things for humanity?” David practically spit the words at Mike.
Mike stopped, and looked around at the group. “You know I felt terrible when I thought my father was dying. I feel terrible for Bill Larry. But those events were in the very first days after ELOPe…” Mike hesitated, searching for words. “After ELOPe was born. Think about young children who want to get their way. They yell, they hit people. They act in inappropriate ways because they lack the knowledge that some behavior is and isn’t socially acceptable, and they lack the experience and sophistication to understand alternatives. ELOPe was young. That doesn’t make what happened any less wrong, but it does suggest that ELOPe may have grown out of that phase.”
David’s face grew red, and he looked ready to launch another attack on Mike. Mike uncomfortably looked away. Sean put one arm on his shoulder, and forced him back into his seat.
“Calm down everyone,” Sean said, looking at each person in turn. “You’re all tense, angry, frustrated, and with good reason. We have the welfare of a multibillion dollar business, the free will of the world, and the future of humanity at stake. No small stakes.”
Despite his own anger, David looked around, noting the tension on people’s faces. Rebecca had a wisp of her hair broken loose, something he’d never seen before.
“I’m not sure we would be able to stop ELOPe, even if we tried,” Sean said, slowly and carefully. “We made a solid plan to bring it down. Some of the most brilliant people in the world work here, and we had them work on this problem. We had several options on the table for how to deal with ELOPe and we took the most thorough, most aggressive option available to us to eradicate it. If what you’re saying is true, David, then we weren’t effective.”
Gene sat quietly, looking at the table, but shaking his head in silent rejection of Sean’s speech. He had the look of a man who didn’t like where the conversation was going.
“Now we have to step back and think about this situation. Out there, people have been fighting a losing war against ordinary computer viruses for years,” Sean continued, gesturing toward the window. “Now we have what is effectively the smartest virus that’s ever existed. Not only can ELOPe exploit every computer trick available to it, but it routinely engineers people into giving it what it wants. ELOPe can learn and adapt, and hire humans to make improvements to it. It’s understandable to fear what it can do. And we can be sure that if ELOPe was taking precautions before against being removed from servers, then after our attack, it will have redoubled its efforts to ensure survival.”
As Sean slowly circled the table and spoke, David struggled with his emotions. It just didn’t matter what Sean was saying. He knew in his body that ELOPe was wrong. An abomination that would rob mankind of the right to make their own choices. It was impossible to even consider allowing ELOPe to exist. But, in spite of this, his respect for Sean kept him quiet.
Sean paused, and paced in front of the window. “Don’t get me wrong. I’d still like to eliminate ELOPe from the wild, if we could,” he said quietly, almost talking to himself. “Of course, I’d love even more for Mike to be right and to discover that ELOPe is truly helping us, becoming a benevolent caretaker of the human race. But regardless of either of those scenarios, I’m simply being pragmatic here when I said that unless we as a society give up computers entirely, we may never be able to get rid of it. Unfortunately, giving up computers is impossible. Modern civilization would simply stop if we turned off every computer. It’s not like we’re talking about the inconvenience of being unable to email someone. Payments couldn’t be processed, machinery couldn’t run. We’d be unable to make phone calls, or access business records. Business activity would deadlock. Cities would likely be uninhabitable, as the support services would fall apart: food, water, sanitation. That’s fifty percent of the world’s population at risk.”
Sean turned back to the group. “That’s not even the worst problem. If we become too much of a threat to ELOPe, then it will take more active steps against us. If ELOPe was actively fighting humanity, who knows what might happen? At the minimum, we could cause civilization to crash for a few years. Most of the city dwellers would die, all the developed world would decay into anarchy. In the worst case scenario, we could be talking about the extinction of humanity. Could you imagine all the military’s autonomous fighting vehicles in the control of an A.I.?” Sean slowly shook his head.
“We need to leave ELOPe alone,” Sean concluded firmly. “We can closely, discretely monitor it. But any further hostile action is almost certain to fail, and will create a great risk of retaliation.”
The sage of Avogadro had spoken. David was flabbergasted by what he heard. He had come into the room expecting full support for any measures that needed to be taken, and now his best friend had taken the side of the AI, and the smartest person at Avogadro had just said they shouldn’t bother to try because they couldn’t hope to win. But he wasn’t giving up without a fight. He got to his feet, and started yelling.
David continued to argue for fighting against ELOPe, and Gene fought with him, but they lost the battle with the other executives. With Sean’s decision, the executive team was unified. David and Gene grew more strident and their voices louder, until Rebecca yelled for them to be silent.
“Listen closely,” Rebecca said, “because I’m only going to say this once.” She stared pointedly at David and Gene, who withered under the intensity of her gaze. “You two are not going to actively oppose ELOPe in any way. You are not going to say anything to anyone about this without permission. As far as we’re concerned, the problem is solved. If you try to take this information public in any way, it’ll be the last time you work in this industry or any other. Nobody will believe you. I’ll make sure of it myself.”
Sean gestured for Rebecca’s attention.
“Yes, Sean?” Rebecca said, never taking her gaze from David and Gene.
“I agree that we’ve got to keep this absolutely contained. We need a small, very small team to monitor ELOPe. Perhaps myself and two or three others. For everyone else, we can tell them the eradication plan worked.”
Kenneth nodded his agreement.
Sean didn’t verbalize it, but he secretly harbored the assumption that the real force that would keep David from spreading the secret would be ELOPe itself.
David finally could take no more. He opened his hands pleading. “Please. This is one dark secret you’re going to try to keep. One day humanity may look back on you and put you in the ranks of Hitler and Stalin. How will you live with it every day of your life? You can’t make this decision.”