“Anyway,” Kuroda said, “George Zipf was a linguist at Harvard. In the 1930s, he noticed something fascinating: in any language, the frequency with which a word is used is inversely proportional to its rank in a table of the frequency of use of all words in the language. That means—”
You don’t have to spoon-feed Calculass! “That means,” she said, “the second most-common word is used one-half as often as the first most-common, the third most-common is used one-third as often as the first most-common, the fourth most-common is used one quarter as often, and so on.” She frowned. “But is that really true?”
“Yes,” said Kuroda. “In English, the most-common word is ‘the,’ then ‘of,’ then ‘to,’ then … um, I think it’s ‘in.’ And, yes, ‘in,’ or whatever it is, is used one-quarter as often as ‘the.’”
“But surely that’s just a quirk of English, isn’t it?” said Caitlin, shifting slightly on the table.
“No, it’s the same in Japanese.” He rattled off some words in that language.
“Those are the four most common, and they appear in the same inverse ratio.”
“And it’s true for Hebrew, too,” said Anna.
“But what’s really amazing,” said Kuroda, “is that it doesn’t apply just to words. It applies equally well to letters: the fourth most-common in English, which is O, is used one-quarter as much as the first most-common, E. And it applies to phonemes, too — the smallest building blocks of speech — and, again, in all languages, from Arabic to…” He trailed off, clearly trying to think of a language that started with Z.
“Zulu?” offered Caitlin, deciding to be helpful.
“Exactly, thanks.”
She thought about this. It was indeed pretty cool.
“Everything Masayuki said is right,” Anna said, “but you know what’s even more interesting, Caitlin? This inverse ratio applies to dolphin songs, too.”
Well, that was awesome. “Really?” she said.
“Yes,” said Kuroda. “In fact, this technique can be used to determine if there is information in the noise any animal makes. If there is, it will obey Zipf’s law, so that if you plot the frequency of use of the components on a logarithmic scale, you get a line with a slope of negative one.”
Caitlin nodded. “A line going diagonally from the upper left down to the lower right.”
“Exactly,” said Kuroda. “And when you plot dolphin vocalizations you do get a negative-one slope. But if you take, say, the sounds made by squirrel monkeys, you get a slope, at best, of -0.6, because what they make is just random noise. Even the SETI people — Search for Extraterrestrial Intelligence — are doing Zipf plots now, because the inverse-relationship is a property of information, not of any particularly human approach to language.”
All right, all right: it was cool math.
“Now do you see why I like information theory so much?” Kuroda said, his tone suggesting he was still trying to cajole her. “Hey, do you know John Gordon’s old story about the student of information theory on his first day at university?”
Anna said, “Not this one again!” but Kuroda pressed on undaunted.
“Well,” he said, “the student shows up at the departmental office and hears the professors calling out numbers. One would call out, say, ‘74!,’ and all the other professors would laugh. Then another would call out a different number, say, ‘812,’ and again everyone would laugh.”
“Uh huh,” said Caitlin.
“So the student asks what’s going on, and a prof says, ‘We’re telling jokes. See, we’ve all worked together so long, we know each other’s jokes by heart. There are a thousand of them, so, being information theorists, we applied data compression to them, assigning each one a number from zero through 999. Go ahead, try it yourself.’ And so the student calls out a number: ‘63.’ But no one laughs. He tries again: ‘512!’ Nothing. ‘What’s wrong?’ the student asks.
‘Why is no one laughing?’ And the kindly old prof says, ‘Well, it’s not just the joke — it’s how you tell it.’”
Caitlin found herself smiling despite herself.
“But one day,” Kuroda said, “the student was looking at a weather report for the far north and happened to exclaim the temperature: ‘Minus 45!’ And all the professors burst out laughing.”
He paused, and Caitlin said, “Why?”
“Because,” he replied, and she could tell by his voice that he was grinning, “they’d never heard that one before!”
Caitlin laughed out loud, and found herself feeling better, but her father said, “Ahem” — actually saying it as if it were an English word, rather than like a throat-clearing. “Might we get on with it?”
“Sorry,” said Kuroda, but he sounded like he was still grinning. “Okay, here we go…”
He used the technique he’d developed before to send freeze frames of the Jagster data to Caitlin’s eyePod, and from there to her implant. By trial and error, they found the right refresh rate to get what she was seeing to increment by just one step — just one iteration of whatever rule was governing the cellular automata as they changed from black to white or vice versa. She could now watch, frame by frame, at whatever playback speed she wished, as spaceships moved across her field of view, without missing any steps.
Kuroda had no way to filter out just the cellular automata from the Jagster feed, but Caitlin could do it with ease, simply by focusing on only a portion of the background.
“And,” he said, “speaking of Mathematica, Malcolm, do you have it?”
“Of course,” he said. “It should be accessible here. Let me…”
Caitlin heard them moving around, then, after a bit, Kuroda said, “Ah, thanks,” to her dad, and then, generally, to everyone, “Okay, let’s run the Zipf-plotting function.” Key clicks. “Of course, we’ll have to try a lot of different ways of parsing the datastream,” he continued, “to make sure we are isolating individual informational units. First, we’ll—”
“There!” interrupted her dad, actually sounding excited.
“What?” said Caitlin.
“Well, that’s it, isn’t it?” said Kuroda.
“What?” she repeated more firmly.
“You’re sure you’re concentrating on just the cellular automata?” Kuroda asked.
“Yes, yes.”
“Well,” he said, “what we’re getting as we plot them flipping from black to white is a lovely diagonal line — from the upper left to the lower right. A negative-one slope all the way.”
Caitlin lifted her eyebrows. “So there is information — real content — in the background of the Web?”
“I’d say so, yes,” said Kuroda. “Malcolm?”
“There’s no random process that can generate a negative-one slope,” he said.
“Le’azazel!” exclaimed Anna; it sounded like a curse word to Caitlin.
“What?” said Kuroda.
“Don’t you see?” Anna said. “A negative-one slope: it’s intelligent content on the Web in a place it’s not supposed to be — intelligence disguised to look like random noise.” She paused as if waiting for one of the men to supply the answer and, when they didn’t, she said, “It’s got to be the NSA.” She paused, letting that sink in. “Or maybe it’s comparable spooks elsewhere — Shin Bet, perhaps — but I’d bet it’s the NSA. We already know, from Hepting, that they muck around with the traffic on the net; it looks like they’ve found a way to package clandestine communications that move in the apparent noise.”
“What sort of content could it be, though?” asked Caitlin.
“Who knows?” said Anna. “Secret communiqués? Like I said, people have tried to use cellular automata before for data encryption, but nobody — at least not anyone who’s gone public — has ever worked out a system. But the NSA scoops up a lot of the top math grads in the US.”
“Really?” said Caitlin, surprised.