“You’d be afraid of your own shadow if someone told you Aldous Gibson invented it.”
Paine managed a faintly amused grin, but it melted when he briefly considered that it might be his last. “You know, Doc, I think you’re right about that. I’d think twice about anything that Gibson created, which brings me to my reason for this chat.” Paine held out one of his cybernetic arms and gestured toward Sanha. “Your A.I. has already told you that this is Professor Sanha Cho, a former post-human. What your A.I. hasn’t told you—what it didn’t know—what I didn’t even know until twenty minutes ago—is that Professor Cho is the one who gave us the location to the post-human facility.”
“He’s right,” the A.I. said, his voice tinged with surprise. “This is entirely unexpected.”
“So he’s a traitor,” Craig observed. “So what?”
“Not a traitor,” Sanha replied defensively. “A man that was willing to give up everything for a chance at peace.”
“Give up everything?” Craig responded. “That’s funny, considering you’re the only post-human who’s still alive. Seems like you’re the only one who didn’t give up a damn thing.”
Sanha looked up apprehensively at Colonel Paine, like an abused animal seeking its owner’s permission to step away from its leash.
Paine tilted his head toward Craig, urging Sanha to continue.
“I-I didn’t know they’d kill everyone. That’s not what I intended.”
Craig shook his head in frustration and closed his eyes as he flexed his large and powerful hands. He wanted to put them around Sanha’s throat and start squeezing; he didn’t think he’d ever let go if he got the chance.
“This war—this conflict—was never about A.I. or no A.I.,” Sanha began to explain. “It was always about control. Power. Absolute power—and who would have it. Gibson or Morgan.”
Craig turned back to Sanha, his eyebrows knitting. “What are you talking about?”
“The A.I. hasn’t told you how it came to be, has it?” Sanha asked.
“I haven’t had time to relay my origin to you, I’m afraid,” the A.I. said to Craig.
“It was grown,” Sanha revealed, “just like a person would be grown—only much more quickly.”
“What do you mean, ‘grown’?”
“The A.I. doesn’t have a brain that emulates the architecture of a human brain. The truth is, we still don’t understand everything about how a brain works. Aldous solved this problem, as the Chinese did before him, by employing a cognitive science-based, explicitly goal-oriented strategy when developing the A.I. In other words, he designed programs that could combine virtual neural patterns together to form new, random patterns that would then be tested to see if the patterns had the desired qualities. Evolution does the same thing when two parents come together to form offspring. Some are successes and others are failures, and more often than not, the successes combine with other successes to produce even more desirable offspring. But, while evolution takes millions of years, virtual combinations are infinitely faster. The A.I. was built this way—the outcome of high-speed computer evolution.”
“His description is accurate,” the A.I. confirmed for Craig.
“All right. So?” Craig asked.
“The A.I. wasn’t the only program to be created in this manner. Aldous also designed virtual worlds where the A.I.’s could be tested. They were given autonomy within the confines of these worlds and then tested one last time in an apocalyptic scenario that they thought was real. The A.I. inside of you right now is the only A.I. that passed the ultimate test.”
“And what was that?”
Sanha smiled. “Ask it.”
“I was willing to sacrifice myself to save humanity,” the A.I. replied.
“So you’re telling me that the A.I. proved it’s a good guy. If that’s the case, why are you trying to destroy it?” Craig asked.
“I’m not trying to destroy it,” Sanha replied, “and neither are the Purists. They’re trying to use it.”
Craig turned to Paine with an expression that silently asked for confirmation of what Sanha was saying.
“He’s telling the truth. We don’t mean you or the A.I. any harm.”
“Do you expect me to believe that?” Craig asked. “Your government ruined the world over your belief that A.I. is evil, and now you’ve just…changed your minds?”
“We don’t really have a choice anymore,” Paine replied. “The current global situation is unsustainable. When we struck against the Chinese A.I. fourteen years ago, strong A.I. was something it took the resources of an entire nation to realize. Now, all it takes is a few super processors and a small team of people with the right amount of human ingenuity. Aldous and his team were the first to succeed, but they won’t be the last. We’re fighting a losing battle.”
“Humans just can’t monitor everything,” Sanha added. “The Purists have finally figured that out. It’s not practical to try to stop the exponential advancement of technology and, as technology advances, it becomes possible for small groups and even individuals to do greater damage with cheaper and more accessible resources. There was only one sustainable solution to the problem—nannification.”
“What?” Craig reacted.
“Creating an A.I. Nanny.”
“What?” Craig repeated, this time even more perplexed.
“Basically, an A.I. Nanny is an intelligence that is superhuman, but only mildly so—above us the way we are above higher order apes. It would be tasked with protecting the human species from ourselves. The A.I. could provide stability, and it would have control over a worldwide surveillance system so it could monitor everyone who is online and make sure no one else is trying to build a competing A.I. that could become malevolent. It would control a network of robots in the service industry and be in charge of the world’s manufacturing. It would even control traffic with self-driving cars.”
“So why are the Purists willing to go along with this idea now?” Craig asked. “They could’ve done this all along.”
“Aldous Gibson wasn’t the only one who was determined to build a strong A.I.,” Paine replied. “We’ve intercepted hundreds of other less sophisticated attempts at various stages along the process. Some of them were dangerously close to success—untested, unregulated, extremely versatile A.I.s that were less than six months from coming online and wreaking havoc. If you think WWIII was bad, imagine a malevolent super intelligence running free, exponentially augmenting its own intelligence. Humanity wouldn’t stand a chance.”
“So you’re trusting Aldous’s A.I. just because it passed a test?”
“No,” Sanha answered. “The virtual scenario was a large part of it, that’s true, but there’s more. It is preprogrammed with a set of goals. It has an inhibition against changing its programming. It won’t rapidly modify its general intelligence, and it’s even been programmed to hand over its control of the world to a more powerful A.I. within 100 years. It will see it as its mission to abolish human disease, death, and our current economy of scarcity so clean water, power, food, shelter, and everything else we need will be abundant. And, most importantly, it will prevent the development of technologies that might block it from carrying out its overall mission, which is to improve the quality of human life, without ever taking actions that a strong majority of humanity would oppose.”
“Seems like you’re putting all your eggs in one basket, Professor,” Craig observed.