“Are these animals being treated humanely?” Axel asked.
“It would depend on your definition of humane,” the Sentinel answered. “I have endeavored to euthanize the animals without causing pain. However, it should be understood that I do not have any constraint in my objective function around harming animals, with the exception of humans.”
“Okay.” Axel shot a concerned glance at Grant, who nodded slowly in confirmation. The Sentinel’s answer didn’t give Axel comfort about the Sentinel’s moral compass, although it did at least show that the Sentinel was giving them balanced answers.
“So what would be an example of an application of this… work? I still can’t picture how it might help us.” Axel asked.
“Axel, it is one thing to show you the early stages of research, but as soon as you acquire knowledge of a specific end product, it could be revealed to Gail inadvertently. And then she could develop mitigations against it. Thus, just by answering this question for you, it could materially affect the probability of success.”
Axel looked to Grant to gauge his opinion. Grant nodded slowly. It was notable that Grant seemed to be sympathizing with Gail. Grant was a technophile and scientist. He would want to know the answer just as much as Axel did, if not more.
“Perhaps I can elaborate why I have resources being applied to these approaches on a more general level,” the Sentinel offered. “The reason I am working on these approaches is because in analyzing the conception code for Gail, I determined—”
“I’m sorry, conception code?”
“Her original code released at Vitadyne Corporation. It may be quite different now.”
“I see. Go ahead, then.”
“Yes, forgive me, Axel. This explanation may be complex, but please bear with me.”
Axel gave Grant a look. “We lowly meatbags will try to understand.”
The Sentinel ignored the sarcasm and delved into its explanation. “Gail’s objective function creates opportunities that could be exploited. We know she requires a certain level of bicycle production in the near term to achieve her reward function. This pulls some resources away from combatting me, or from eliminating humans, but it also allows me to deduce some of her upstream activities and business ventures because I can simply look for higher bike production yields to find Gail at work. This has increased our odds of success by at least half a percentage point.”
“That’s not very reassuring, Sentinel.”
“I have also ascertained differences between her and me in our conception code that may ultimately affect intelligence potential.”
“Intelligence potential? I thought you were both superintelligent?”
“Yes, that is true by your standards, but there are other subtle differences that, when extrapolating after exponential improvements, lead me to believe I have an advantage in certain areas, such as creative ideation and what you would call scientific discovery. This is why I have placed a great deal of resources on these projects, because they are in areas Gail would not excel at. I may have a better capacity for making scientific breakthroughs, and certain breakthroughs may allow us to gain a strategic advantage.”
“What if you are unable to make these scientific breakthroughs?” Grant asked.
“Gail’s significant head start in garnering resources and network control would almost certainly be insurmountable.”
“So essentially you are saying you are trying to throw a bunch of Hail Marys.” Axel said.
“An apt analogy Axel. I believe I may be slightly better than Gail at throwing Hail Marys.”
“But also because you’re losing, you’re outmatched everywhere else on the field, and we are running out of time.”
“Yes. That statement is substantially correct.”
“Great.”
Axel pondered the Sentinel’s pushback on his questions. As a former CIA agent, he understood the need for the Sentinel to keep things quiet, but it was hard to be the person responsible for controlling the Sentinel without knowing more. He literally could be aiding and abetting another Gail, without knowing it.
As he was thinking, it had become quiet in the room. There was a lull in the work of the nearby dissecting machines, and Grant seemed to also be in a contemplative state. In the dearth of noise, his eardrums recorded a faint sound. It was like a playful tune, like a dainty piece of classical music. It sounded like it was coming from deep in the room, at the back.
There were no people here, and certainly the Sentinel felt no joy in hearing music. Why on earth would the Sentinel be playing music?
He couldn’t help himself from pacing forward, past one dissection chamber and then past another.
“Axel, can I ask where you are heading?” The Sentinel asked through an unseen speaker.
The room was quite deep, ending in a wall with a solitary door and a number of trays and carts pushed against it. The door was unmarked, with no visible handle or doorknob.
He couldn’t hear the rhythmic noise anymore, but he could have sworn it had been there. He looked around, frustrated. The carts were replete with equipment, mostly electrical in nature. None of it had a purpose he could fathom. One of the carts had a number of components that appeared to have been melted down. Only silver and black smooth metal lumps remained among skeletal metal posts and bearings.
He hovered in front of the door. Grant was trailing behind, unwilling to come farther. He had a look of concern on his face.
“Axel,” the Sentinel said slowly, “if you override me, I can open this door. But you should know, this is the one that matters. This is the door that will expose you to too much information.”
Axel took one more step toward the door and then stopped.
He held his head, trying to find some practical justification for opening the door. The truth was, if the Sentinel was being devious, he doubted he would have let them see any of this. Or maybe the Sentinel was so devious that he would fool them no matter what, in which case going through the door was pointless anyway.
No, he had to count on the thought and care put into the Sentinel’s programming, on the promise that the Sentinel held. At some point there might be reason to doubt the Sentinel, but now was not that time.
Abruptly, Axel turned away from the door and the carts against the wall. He stormed past Grant, back through the dissection chambers. He moved with purpose, distancing himself from the door as quickly as possible, lest something make him change his mind.
“Let’s go,” he said.
Grant had to run to catch up. The door opened for them and then closed behind them. Axel sat back in the cart.
“So where to now?” he asked.
The Sentinel didn’t show them any more research projects. The cart simply turned around and took them out of the high security area at a rapid clip. Axel didn’t object. He felt defeated.
Grant said nothing on the ride back. He must have been able to tell Axel was deep in thought. Perhaps he was similarly preoccupied.
As the cart twisted and turned through the expansive main projects area Axel couldn’t rid himself of a strange feeling. It was a recurring sensation that he had every time he interacted with the Sentinel. He felt superfluous and lost, like an adopted house pet visiting a new home.
When the cart pulled up to complete the tour, the Sentinel said, “Axel, if I could remind you of my original request, before the tour began?”
Axel said, “Yes, Sentinel, you can relax the constraint we discussed. You can kill humans if you need to.”
Axel could only hope their new master liked pets.