Gray got out, but leaned back inside before departing. "Ask Margaret to pull up the back prop reports." He then took off, running to catch up with the departing worker.
Laura stuck her head out the car's open door. "Excuse me!" she called out to a worker. When he walked up to the car, she said, "What's going on?"
"O-o-oh, we just had a man up and quit."
"Quit? Why?"
He shrugged. "Said it's getting too dangerous in there. Too many malfunctions."
"What kind of malfunctions?"
"Well, it's kinda hard to put your finger on it. It's just a feelin' you get, you know? They're misbehavin'. They're actin' like they got somethin' more important to do than work."
"I'm sorry, but I don't understand. Who's misbehaving?"
"You know… the robots."
24
"What do you want with old back prop reports?" Margaret asked in a harried and barely civil tone. Scraps of hand-scrawled notes littered her semicircular desk between half a dozen monitors.
"Gray wants me to look at them," Laura replied. "I don't even know what the hell they are."
Margaret sighed impatiently. Her eyes were hidden behind eyeglasses bathed in the colorful glare from her computer screens.
"They're old reports we used in the early days to keep track of the net's progress. Every night, after the staff went home, we'd turn the net loose and let it pose its own questions and come up with its own answers. The next day we'd strengthen or weaken the connections depending on whether its conclusions were right or wrong."
Laura shrugged — at a loss to see how that might help her research. "What do the reports look like?"
"They're organized into conclusions and analyses. It's all real easy to read. Gray wanted to use grad students from non-technical disciplines to check them. They'd pour over the reports and enter a strength weighting from one to ten, one being the least accurate conclusion and ten the most accurate." Margaret laughed, which in her seemed to come off as derision. "Every day some English lit student would come tearing into my office shouting, 'It's alive! It's conscious!' I'd take a look, and the computer would've made some bullshit conclusion like 'Men aren't attracted to hairy women.' It would support the conclusion with analysis like 'Hair is a sign of older age and consequently shorter reproductive life.' Mr. Gray and I used to have a good laugh at closing time going over those monumental discoveries."
"So you and Mr. Gray went over those students' findings even though they were no big deal?"
Margaret had returned to her work. She reluctantly looked up at Laura. "Mr. Gray, as you might've heard, doesn't need much sleep. He'd spend most evenings at the lab doing minor housekeeping. Going back over low-priority things like aberrations."
"What aberrations?"
"Abnormal conditioning. There's an intrinsic risk in allowing the net to draw its own conclusions. It goes down rabbit trails into unproductive knowledge domains. Things like…" She thought for a minute. "I remember one. 'People lacking an ability to empathize are capable of horrific crimes. Analysis: They can't appreciate that their acts of convenience are the tragedies of their victims.'" She laughed.
"That one sent some doctoral candidate from the sociology department right through the roof."
"But that's… that's brilliant!" Laura said.
Margaret looked at Laura with a smirk.
Laura ignored the intended slight. "So you'd go home and Mr. Gray would stick around at night to review these 'aberrations' you found?"
"Yes," Margaret replied curtly.
"And he would do what with them?"
"He would decondition the connection, of course," Margaret shot back with unexpected vehemence.
Laura remembered something Gray had said about Margaret. She went home to her family every night like clockwork. That would have left Gray alone in the lab with the computer.
"I'd like to see the back prop reports," Laura said.
"All thirty billion of them?" Margaret replied, then laughed.
She shot her thumb toward the door. "Just ask one of the techs."
"Thirty billon? How could you possibly check all those reports for errors?"
"We couldn't. We just sampled a small fraction of them."
It took what seemed like forever, but a technician finally loaded a program called a "browser" on the computer in Laura's office. He pulled up a hundred back prop reports as a test. When assured all was in order, he left Laura alone at her desk. She read the first report and found it interesting — a budding mind at work organizing itself.
The next few reports were more of the same, as were the next, and the next, and the next. Tons of minutiae, all parsed into tidy logical arguments.
Most revealed the computer's difficulty interacting with the physical world.
<Conclusion: Grip tension should be increased when speed of end effector is accelerated. Analysis: Inertial force is resisted by traction, which can be added by tightening pressure on surface. But see: greater risk of structural failure when tension of grip is increased.>
Laura yawned and read on.
Out of the mass of mundane conclusions, a few stood out from the rest.
<Conclusion: Expression "time's a-wasting" is preferable to "Time is money" as polite means of encouraging haste. Analysis: "time's a-wasting" is considered a gentle prod, while "time is money" is a comment upon the time value of money. See: net present value, net future value, and discounted future net cash flow.>
<Query: What does "Time is of the essence" mean?>
While the report conclusion was correct, its analysis contained an obvious error. The computer connected "Time is money" with what it knew about financial calculations instead of appreciating the saying's subtler meaning. What interested Laura more, however, was that the question was asked at all. The computer was clearly struggling to assimilate to adopt the "knowledge domain" of human culture.
And what's the difference between that and actually becoming a human? she wondered.
Laura glanced nervously at her watch. She had only twenty minutes to advise Mr. Gray whether to load the phase-three. If the computer's problem was a virus, the phase-three might be the only thing that could save it. But if the computer was emotionally disturbed, the vicious antiviral program might destroy the dense maze of conclusions and analyses that was the machine's brilliance.
A red bar along the top of the screen set out a menu of the program functions. One was entitled Search. She'd told Gray she needed to know more about the computer's "emotional database," and he had directed her to the back prop reports. Laura clicked on the Search command. In the query box she typed "love, hate, fear" — she looked up at the ceiling in search of more words—"parent, child, lover." Laura clicked on the button labeled Go.
The cursor flashed and flashed and flashed. "Come on," she urged.
<927,964 entries contain words "LOVE, HATE, FEAR, PARENT, CHILD, or LOVER.">
"Oh, ma-a-an," Laura mumbled. She was certain she'd made some mistake in phrasing her search request. Once, back at Harvard, she had accidentally searched the Web for any articles with the word "disassociation" or a comma in them. She'd had to unplug her computer to make the machine stop searching.
She pulled up the first report with a sigh.
<Conclusion: When parents intentionally cause grievous bodily harm to their child, they are expressing either love or hatred.>
<Analysis: Indifference does not typically engender violence.>
"Jesus Christ," Laura muttered. She skipped to the next report—<No. 2 of 927,964.>
<Conclusion: If someone's personality is dependent, but that person's lover has an independent personality, the dependent person often becomes abusive.>