“Honestly? Yes. We had this conversation, word for word, last week.”
The cursor blinked. Iceweasel was convinced this was for dramatic effect. Dis could scan the logs of all their conversations in an eye-blink, but when something emotionally freighted happened, there was a blinking delay. Iceweasel thought it was Dis’s lack of a body’s expressive range. She found herself interpreting the blinks – this one is a raised eyebrow, that one was a genuine shock, the third was a sarcastic oh-noes face. There were pictures of Dis’s human face in all these expressions and more – stern and lined, with dancing blue eyes; thick, mobile eyebrows and a hatchet-blade nose – but when Iceweasel thought of Dis’s face, she thought of that cursor, blink, blink, blink.
“So we did. Depressingly enough, I figured that it was a groundhog day moment at this point. I must bore you.”
“Not usually. I sometimes try out weird conversational gambits at these moments, to see how different your responses are. This is one of them, incidentally.”
Computer laughter was weird. Iceweasel felt a child’s pride in coming up with a joke that made her parents crack a smile. Dis’s laughter echoed through her earphones. “What’s your hypothesis? If I say the same thing no matter how you react, am I more or less of a person than if I vary my responses based on input? Conceptually, it doesn’t seem like either one would be harder to simulate – both are chatbot 101. We both know plenty of read-only people who always say the same thing no matter what we say.”
“I think you’re optimizing yourself to be tunnel-vision fixated on the cog-sci sim work and you’re incapable of getting off the subject.”
“I see that we’ve done variations on this before.”
“Yeah.” Iceweasel didn’t add, and then you melt down.
When Dis told her about groundhog daying, named after the old movie, she’d underestimated the way the experience would play out for her, going through the same conversations over again, trying different gambits but ending up in the same place, with an incoherent, crashing sim.
“The literature on this is drawn from brain injuries, temporal lobe insults that nuke short-term memory. The videos are weird: every couple minutes, some old lady has the same conversation with her nurse or daughter: ‘Why am I in this hospital?’ ‘I’ve had a stroke? Was it bad?’ ‘How long have I been here?’ ‘What does the doctor say?’ ‘What do you mean, my memory?’ ‘You mean I’ve had this conversation with you before?’ ‘Every ninety seconds? That’s terrible!’ and then back to ‘Why am I in this hospital?’ Around we go.”
“Well, your loop lasts more like a day, and isn’t that banal.”
“You say the nicest things.”
“It’s interesting to see the differences between reboots. I can’t get over how cool you are with the idea of being annihilated between reboots. You can access the logs, but you wake up knowing that you’ve had a day wiped off the books, and it never slows you down. I get that you’re able to control that, but...”
“You really don’t understand. No offense. Back up to that read-only person who always answers the same: the reason that person is so frustrating is we know that people can change based on what they know. You’re not the same person you were when you got here ten days ago. If I asked you-minus-ten and you-now the same question, you wouldn’t be surprised if you gave a different answer. If I asked a battery of questions, you’d be surprised if you didn’t give different answers. The you that is you is actually the space of things that you might think in response to some stimulus.”
“The envelope.”
“You know this, but you don’t. When I come up clean, I’m only allowed to come up within the section of the envelope that doesn’t freak, which we can find, thanks to the lookahead. Imagine how life will be when everyone gets scanned regularly, when we build bodies that we can decant sims into to bring them to life. There’d be social pressure to not sweat the idea that it’s not ‘you’ in the sim, and anyone who suffers meat-death and comes back as a sim will only be brought up in the corner of the envelope that doesn’t freak and suicide. Give it a generation and there won’t be anyone alive cognitively capable of an existential crisis. I’m a fucking pioneer. Partly that’s because I’ve had years to get used to the idea that everything that makes you recognizable happened in the interactions of physical matter in your body, following physical rules from the universe.”
“I have a friend back at the B&B, a real hard-line walkaway. She’s always talking about how she’s not a special snowflake. I bet she’d love that: ‘you’re just meat following rules.’”
“Well, if you’re not meat following rules, what are you? A ghost? Of course you’re meat. The way you feel is determined by your gut, the hairs on your toes, your environment. I don’t have those things, so I am feeling differently from when I was meat. But when I was meat and forty, I felt differently from when I was meat and four. I have continuity with meat-me, what it remembered, that’s enough.”
Iceweasel’s eyes flicked to the timer. Dis’s cameras were acute enough to spot it. “I’m overdue for my four o’clock meltdown.” She’d had thirty hours of uptime and Iceweasel slept in fits, doing an hour or two of chatter with Dis for every three hours Dis spent with the researchers.
“You’re making progress. The work must be coming along.”
“You’re selling yourself short. The only person making progress around here is you, chickie. You play me like an organ. I watch your eyes when we’re talking, see you keeping track of my equilibrium, steering the conversation to keep me between the lines. I don’t know if you know you’re doing it. You’ve become the world’s greatest bot-whisperer. It was inevitable. Any time you give someone feedback and tell them to control it, their brains will find patterns in the system and optimize them. You’ve done it as sure as if I’d put you in a sim and written an app for your subconscious.”
Iceweasel felt her neck prickle. Dis was scary-smart, literally inhuman. Every now and again, Iceweasel had the impression she was being manipulated by the sim. “I thought you were going to say that it was my people skills.”
“Okay,” Dis said. “Raised by zottas, so you got a dose of the psychopath’s ability to make people want to like you even as you’re screwing them.” Back at the B&B, Iceweasel became expert at deflating criticism based on her rich parents. Dis treated it with the matter-of-fact brusqueness with which she conquered every subject. Nothing Iceweasel said made a dent in Dis’s rhetoric. “You hate it when I talk about your money,” Dis said. The sim had lots of cameras, and cycles to evaluate their data.
“No, I love being judged by my parents. Zottas are the only people it’s okay to be a racist about.”
“It’s not racism when you’re discriminated against for your choices.”
“I chose to walkaway.”
“But you identify enough to get shitty when I pass comment on their social tendencies.”
Iceweasel looked at the clock again. Dis busted her.
“Don’t worry, I’ll melt down soon. I’m feeling it. There’s something not right. I feel it from the moment I come up, like hamsters running on a wheel in here, chased by something they can’t see but know is in there. It’s hard to name, but the longer I’m up, the closer it comes—”
“It’s the no-body thing.” Iceweasel felt a shameful spurt of joy at being able to turn the screw on the sim.
“Fuck. I’m groundhog daying again.”
“You always talk about how you can never have a body, and even if you get a body, it won’t be your body, and you won’t have continuity with it.”