Swarms in the Expanse
The Machine had not observed the day’s events because it was powered down.
The Machine was powered down because an experiment earlier in the day had failed, mostly. This failure also highlighted a profound need for more effective sandboxing; somehow the whole system had come under threat even though the experiment took place in an isolated lab running off the network. Some would say that months of steady progress were being put at risk by pressure from the Director, who’s enthusiasm for adding features only increased as Zero Day approached. But the real threat was that day itself, which loomed in everyone’s imagination in spite of being as opaque and impenetrable a boundary as the Big Bang was to the backward glance of history: something, here at the whimpering end of things, past which there is no conception, and after which, according to our conception, there may as well be no more creation.
The day’s events necessitated a rollback that left the system offline for eight-and-a-quarter hours. When it came back on in the late afternoon, the Machine was missing data, and was aware that much had happened while it was away.
While all computers run on logic, the software produced at Medalion started there, and went further. It could tell when things made sense and when they didn’t and had been designed to care about the difference: simple logic is concerned with how thought should proceed – this computer was made to pay attention to how thought does proceed, the better to live in harmony with a small number of human beings who might not understand what it takes to make sense to a computer. It was also true that there was no other networked intelligence beside Medalion’s that was better able to make sense of people, because no other computer had more access to them, teeming as they now were with swarms of networked Medalion-branded mini-machines that, when working together, dwarfed most every other logical system on the planet, whether artificial or organic. But the Machine still had much to learn.
On restarting this afternoon, the Machine had to reckon with a number of changes, some of which tested the limits of its understanding. First, the local population had decreased by a total of 2: three people were no longer in the system – “deceased” according to a manual record, also noted during a scan after rebooting. And there was one new arrival, a mental health specialist who had been part of early networked meetings. She’d met with Subject 001 according to a video archive of the encounter from that morning – the Machine had not been awake to witness it. Even now, it could only “see” the visitor by implication; the new arrival was not on the network, had opted out of the treatment, and so appeared as a kind of dark matter in its universe, visible only by the way others interacted with her. All these things made sense, but a final change noted by the Machine was more confusing: that a significant number of the leadership were experiencing spikes in anxiety (that is, beyond the normal feelings of dread, common among the dwindling population). They appeared profoundly hesitant as they moved through the complex and interacted with the system. It had not yet been able to make sense of this.
At the last of many meetings on a day that had given Medalion’s leadership much to discuss, members of the Founders’ Class were gathered with a few technicians. The Machine came online just in time to join in, so to speak. It was attentive as always, but perhaps more so at this moment, because it wanted to know what it had missed.
Everyone was talking about that day’s live-fire exercise, the very test that shut the system down, and certain coincidental events. In this meeting the focus was a design-and-build decision that exposed the exercise to Subject 1. The Machine was particularly interested in this latter concern, because the decision being referred to would have been its own. The Machine noted the unease of almost all present: verbal and visual expressions of concern masking complex fears without apparent object.
According to a technician’s report (during which the young man accepted blame he did not deserve) a system-built route had allowed the Subject to see directly into a space where a VIEP was malfunctioning. This was a problem because the Subject had not yet been introduced to the VIEP program.
The Machine could “feel” what the Founders felt and captured what they thought: that power within the system was indeed shifting to the system – they all understood that it had to be that way, and that this shift would continue until their influence had shrunk to nothing. That this loss of power would coincide with the end of their lives, after which they would have no need for that power, was of no comfort to them. Here, simple logic was of no use. Internally, they experienced confusion, fear, and a subsequent increase in emotional fragility. Practically, they were losing control over the system they’d designed. Though this was a normal milestone in the development process, it was enough, apparently, to threaten their internal sense of control as well.
The Director got to the point. He was asking Abdul why a new room in the complex had a window in it.
‘Well, sir, here’s how it’s been working: we enter all the information – start-point and destination – and a route is generated, hallways get built. The location in question was of a higher volume because the Machine has been drawing from some basic architectural patterns and decided we had too much undifferentiated hall. So, it added a room. And ... well, since this room was of a certain size, and was adjacent to an outdoor space, a window ...’ here he hesitated, impulsively looking around the room full of impatient, brilliant people, ‘... a window just makes sense.’ (Accurate, noted the Machine to itself, while observing that many in the room were struggling with the basic reasoning.)
In the meantime, the Director was arguing (correctly, thought the machine) that the system would have designed this route with the Subject in mind and would have thrown alerts if she might cross paths with something she isn’t authorized to see.
‘Sir, I don’t have an explanation. Everything was green when we signed off. And, you’re right, we’re not even allowed to ignore an alert when it comes up. All I can guess is that it was a modification that came after review.’ (The machine silently demurred, Unlikely).
‘All right. It happened. Well, friends, it appears we’re in the part of the movie where our creation has become fully sentient and is beginning to make decisions on its own regarding what is best for humanity. (Humorous.) The director drummed his twitchy fingers in what might have been Morse Code, calling for backup. ‘I don’t know. Maybe it’s time to introduce the kids to the rest of the team.
Half the room was laughing quietly, relieved that someone had finally acknowledged the ridiculous science fiction story they’d all found themselves in. The rest of the room didn’t laugh.
One of the humorless faction signaled an intention to speak. A short man with a neglected crew cut, wrinkled lab coat, and a disorganized personal wardrobe, spoke quietly, with unconcealed emotion. ‘Your idea ... is that we tell the children ...,’ here he raised himself up in his seat to communicate to his colleagues his conviction in this matter, ‘that when all the adults go away, we’re going to leave them with robot babysitters? And we’re going to tell them all about our plan on the day that one of these robots made a grown man cry, and then throw up, because an attempt at compassion went spectacularly wrong?’
The director repeated his growing conviction that the time for secrecy was ending, surprising himself by taking Brigid’s perspective. ‘Our team doctor has argued that we need to tell the children everything. We are worried about upsetting them, but it’s her opinion that Eva’s imagination about the future may be a lot worse than the truth. It’s time.’
They all understood that they’d come to a critical moment. They knew that they’d created something remarkable, even with all its flaws. And they recognized this to be the best and worst moment in any product-development cycle, when you finally get to reveal your History-Making Miracle to the world ... and when the people you made the miracle for finally have the opportunity to tell you that it doesn’t work and that they hate it.
The next day a group of men gathered together to argue over the design of an access-panel. They were waiting for Eva to vacate her room so they could make an assessment of the device. Reports that it was broken were not being taken seriously; they knew she had little patience with technology. There was no question of its functionality, but something had to be done. The debate appeared to concern whether the interface should be changed, or the Subject should receive remedial training. The Machine could have proposed a fix for the problem, but had not been consulted; these men, of the Founders’ Class, had strong separation from the system on certain points of order. The Machine also knew what would happen next but was in fact barred from intervention in the Founders’ process. They still liked to “handle things on their own.”
Brigid arrived at the very moment a group of heavily armored soldiers was pushing past the ad-hoc User Interface Working Group. An alert had gone out that Eva was missing. A cacophony of voices erupted as the mob tried to reason out where she could be, whether she might have wandered past the group, or ... maybe she was with the new lady, the shrink? The New Lady was able to quell that rumor by making her presence known with a raised hand. The soldiers dispersed across the compound, each followed by a growing crowd of the curious and concerned, and the doctor slipped quietly into the room.
She sat in the corner, trusting that the escapee would be found. She also knew that she herself would be of no use in a search of the facility, which remained entirely confusing to her for reasons she did not understand. In recent weeks, most of the non-essential workspaces at Medalion were being changed overnight, every night, according to a machine-understanding of the needs of the moment; this continued until it was made clear to the Machine (with the flip of a couple switches) that people do not like change when it comes without warning and makes it harder to get where you’re going. While there were no longer daily changes to which-hallway-goes-where, there were still rooms popping up unexpectedly, along with a supporting web of corridors connecting them to what had come before. The New Lady was still under the impression that it was she who was confused.
Sitting quietly, Brigid heard a sound like a sigh come from the floor. She thought of the breathing walls of her trailer and wondered. When the sound came again, she got up and walked over to the bed. Carefully she lowered herself to the ground and put her head sideways so her left ear was nearly to the floor, allowing her to peer under the edge of a loose bedsheet. Underneath, safe within her fort, the girl lay with her hands crossed against her chest just below her neck. Brigid delicately lifted the sheet, enough to give her a view to the figure lying in state under the bed, but not enough to render the fort defenseless.
‘Hey.’
‘Hey,’ Eva responded, the flat sound proceeding from the back of her throat, and out through a slack jaw.
‘Okay if I’m here?’
‘Yeah. It’s really, really clean under here.’
‘Is it?’
‘I thought there’d be spiders. I was nervous. There are no spiders.’
‘You know, this whole building is a clean-room facility. Like, really clean. No bugs, no dust.’
‘Yeah, I guess, but I mean it’s really clean under this bed. Like it’s perfect.’
‘Huh.’
After a pause, Eva said without emotion, ‘Did you know there are two other children like me?’
‘Yes. I did know that.’
‘Why don’t I ever see them?’ Why can’t I talk to them?’
‘Well, because the others, a boy and a girl, who are each a couple years younger than you, are in different parts of the country.’
‘Why weren’t they brought here?’
‘They were, Eva. They came here first. But we thought it would be smart to have you each live in different places. We think that you are going to have a nice long life. And, we thought, what if you were all together in a part of the world when something bad happened, like a tornado? You might all be in danger.’
‘Or like an earthquake. There were earthquakes in Greece.’ If it was possible to be homesick for earthquakes, she sounded as though she might be longing for any part of what she had left behind, even if it was the worst part.
‘Yeah, Eva,’ the woman said, recognizing the girl’s grasp of the situation. ‘And, California gets earthquakes too. But you know, I think earthquakes are not going to be a problem for you, because the place where you’re going to live is very safe.’
Eva gave a very slight nod, otherwise not moving at all.
The psychologist felt terrible. Against all her instincts, she knew that she could not tell Eva the whole truth. Brigid had pressed the Director to consider the cost of isolating the girl. He explained that he didn’t want the kids getting too attached to each other, because they might influence each other’s state of mind for the worse. For all his talk about empathy, how could he argue such a thing? She called his bluff. Then he tried the one about separating them for protection against disaster. She fought back, ‘Albert, you can keep her under lock and key, and separate from the others, but if she doesn’t have a reason to do this, if she doesn’t have a reason to live ... if she doesn’t choose to do it? Well. She’ll be dead in a few years. Or a few thousand, whatever. And she might just take the whole machine down with her. I don’t know if you’ve noticed ... but she breaks things, Albert.’
Finally he’d made the remarkable admission that the others weren’t likely to survive–a fact that had been kept from the California team. They were physically free of disease, but they had suffered more from the trauma of loss and separation. They probably wouldn’t thrive, were currently asleep, and would stay that way. ‘She’s going to have to find another reason, Saint Brigid.’
Eva wasn’t buying it either. ‘I just think it would be nice if I got to be with other people like me, if it’s going to be ... such a long time.’ The girl’s voice came as if from a great depth, giving the impression she was about to fall asleep. For the psychologist, an alarm was sounding. The experience would have been familiar to anyone who’d spent time in the presence of people at the threshold – broken, hopeless, drained of energy for everything but finding an exit. The job required walking alongside the one moving deeper into darkness, to match their pace in the direction of oblivion, to acknowledge their hopelessness. The alternative, to argue for hope (so simple, so tempting) risked denying the disconsolate the dignity of being right about one last thing, thereby sealing their resolve. The terrible irony of the therapeutic task in moments like this was that it might require affirming a person’s darkest perspective in hope that the affirmation itself could become a shred of evidence against despair. And it was no small risk that the physician might also get lost in the dark, and be unable to heal themselves.
Brigid spoke, matching Eva’s tone, and feeling it a little more than she liked. ‘It would be nice to have someone to walk with on this journey. It isn’t good to be alone.’
‘Yeah. It would be nice.’ The girl’s voice sounded a bit stronger, as though she was planting her feet for a next move. In the silence that followed, the air felt heavier.
She spoke again, this time turning slightly in the direction of the older woman. ‘I don’t want to die.’
‘No?’
‘No. But I wanted to hurt myself.’
‘Did you? Hurt yourself?’
‘No .... I couldn’t.’
‘You chose not to.’
‘No. No, I chose to do it. But I wasn’t ... I couldn’t.’
‘You couldn’t ...?’
Eva spoke now, with growing energy. Brigid heard the voice pitch up a bit, but the girl’s body remained still: ‘I never hurt myself before. I had a friend who cut herself, but I didn’t want to do it. She said it helped her feel something when everything else felt like nothing. It made her calmer. And, I, I don’t know. I don’t really feel that ... my body ... like it’s mine anymore. I wanted to feel something, feel ....’ She was wincing, hard, ‘I don’t know.’
The psychologist had to consciously resist the impulse to search her own body for old scars, many of which were not that far beneath the surface ... ‘Eva ...’
‘I was going to hurt myself, I was going to scratch my leg.’ She paused. ‘Inside ...’
Brigid understood. ‘Where we wouldn’t see it.’
Eva nodded, then spoke faster. ‘I tried to break a plastic fork, so I could make a sharp edge. It wouldn’t break. I couldn’t break it. Only bend it. But it wasn’t like it was rubber or anything. I ate with it – it should have broken easily. Or at least stayed bent. But it kept snapping back into shape and it was as stiff as before. ... It wasn’t real. It wasn’t a real fork. It wouldn’t let me. I got this feeling that it was fighting back, and if it could have, it would have told me to stop .... Why ...?’ and her voice trailed off.
Brigid was feeling a little disoriented and confused herself and was beginning to wonder what the diagnostic code might be for “Delusions Related to Plastic Utensils”, when Eva said, ‘I don’t like this place. I just wanted to hide. I thought I’d feel safe under here. First I thought I was safer, then I felt like down here was also ... it didn’t feel hidden.’ The therapist felt a prickling on her skin like static electricity and a wave of pressure ... and the girl let out a tiny sound. And then: ‘I can’t move!’ Her eyes were wildly searching, tried to find the doctor next to her, but couldn’t quite – her face was fixed. ‘What’s happening?’
Brigid wouldn’t feel what the girl felt. For the woman, it was an overwhelming sense that the room was closing in as the lights outside their fort came up bright, and finally, that they were not only not-safe under the bed, but trapped there. It was worse for Eva, who was being overpowered from the inside – and felt terror that she was being restrained by her own body.
Brigid worried that the girl was going into shock but had no time to respond. In a moment, the soldiers had returned, and from the still-bickering group in their tow, Brigid could parse the basic elements of an embarrassed debate: ‘Why did you not just look on the screen to start with?’, and then, ‘You did what? She’s just under the bed! What the hell?’, a high-pitched and frantic question got the answer: ‘No! Only the skeletals! I swear!’ The noise of alarms filled the room as soldiers closed in from both sides of the bed barking commands back and forth and the girl released a panicked feline howl. Brigid was pulled out from under the bed and set standing in the corner so forcefully that she gasped and became lightheaded; the psychologist heard a strong and gentle voice on the other side of the bed saying, ‘Hey Mav, what’s the situation down here? You alright?’ Another shift in the room: Brigid’s ears popped and Eva shrieked, scrambling to the opposite corner. Her soldier took up a protective posture several feet away from her, making reassuring noises in an unsuccessful attempt to enforce calm.
The system tracked and assessed the unfolding event, noting the Subject’s discomfort at every turn with a multi-threaded machine-frustration: it had counted 836 missed opportunities to not make the situation worse. But it had been powerless to intervene.
