Terror in the Skynet
The futile effort to outwit our emotions, and the looming effort to engineer them.
You will stay calm in the face of an emergency.
Lucky numbers 10 15 16 29 31
The four of us—me, my parents, and one of my best friends—were on our way to Venice. My brother was studying abroad there, and we were going to visit him over spring break of my college freshman year. Since all of our hotel rooms could sleep four, my parents offered to bring my friend, who was a few generations removed from being Sicilian, loved pizza, Goodfellas and Spaghetti Westerns, and had always wanted to visit Italy. He was excited for the trip, until we got to the airport.
I wasn’t a seasoned traveler, but had been on a handful of airplanes before. I liked them—the acceleration at takeoff, sudden and ceaseless, without the usual hitches in a transmission’s gearing; the unfamiliar upward pressure rising under your seat; the tilt as the plane rolls through its ascent; the undoing of your mental geography, when, in a time before Google Maps, you see for the first time that that that local landmark is actually there. My friend, however, had never flown before. The thrills I was looking forward to were unknown to him; all he could think was our plane breaking down, blowing up, and falling out of the sky. At the airport, as we walked to get lunch at the food court, I tried to convince him that we’d all be fine.
His fortune cookie disagreed.
Sure, the chances of a plane crash are one in a million—but what are the chances that he’d get that fortune, in this moment, right when I was telling him that no emergency was coming? He thought about going home. I shrugged it off. Even if the Panda Express prophecy was true, I was already a captive to a different emotion. Shortly before the trip, a year-long will-we-or-won’t-we dance with a crush took what I thought was a fatal turn. Flying anxiety was no match for my teen angst: Take us down, I thought. Crash the plane; splash me into the deep blue sea.
Needless to say, we didn’t crash. Rather than having a birthday party at the bottom of the ocean, I spent most of the trip sulking, pumping myself full of Bright Eyes and James Blunt, and telling everyone, "I'm fine," in the most unconvincing tone possible.
Over the next two decades, the story turned upside down. My friend, now one of the best sports broadcasters on TV,1 flies every few days, routinely "commuting" between New York and Mississippi. I went the other way. Far from staying calm in the face of an emergency, I built an emergency out of the calm. Several years after our trip to Venice, on an otherwise uneventful flight from Salt Lake City to Cincinnati,2 something broke loose inside of me. In those three hours, my sense of wonder was replaced by fear, wholesale. Flying became emotionally backbreaking. I began dreading air travel; I started getting nervous about stormy forecasts days before takeoff; I said flight numbers out loud to see if they sounded like future Wikipedia articles.3 I even started to avoid flying altogether—a retreat that, in my fear’s ultimate conquest, drove one of the last nails in the coffin of my eventual relationship with the girl I was sulking over in Venice.4
Does anyone know where the love of God goes
When the waves turn the minutes to hours?Gordon Lightfoot, The Wreck of the Edmund Fitzgerald
As is common for most anxious flyers, it’s the turbulence that gets me. On a smooth flight, I can find an uneasy peace. But add a few bumps—the kind that don’t even get the seatbelt light on, and are inevitable in anything moving at six hundred miles per hour—and I clam up, I can’t focus, and I begin counting down the long minutes until we land. If the turbulence gets worse than that, to say nothing of the shaking that leads to the heart-stopping announcement that flight attendants will need to discontinue their service for a few minutes, I’m overcome, unable to eat or drink, unable to think, frozen in place, my body and mind short-circuited by some primal electricity, as though pinned down and paralyzed by an invisible shock collar.
And so, during most trips, I sit sentry at my window and stitch together a five-hour panorama of clouds and countryside. I stare at the steady ground below, and use it to remind myself the wings are still upright and that physics still work.5 I classify different types of turbulence—the long sways, the gravelly rumbles, the sudden thudding drops, the hard lurches off and back onto a centerline. I monitor for changes in pressure or speed, listen for new engine pitches or hisses in the ventilation, and try to detect hidden connotations in announcements that we've reached ten thousand feet, that Southwest is now offering an exciting new credit card, or that the right engine fell off and we’re pitching straight into Lake Superior.6
Over the years, I’ve collected my preferred set of coping mechanisms. Some are taken from various self-help sites on the internet: (try to) breathe slowly; (try to) count to ten; (try to) relax your shoulders; (try to) plant your feet. Some are chemical: Take sleeping aids; get drunk; take anti-anxiety meds; take someone else’s prescription muscle relaxants. Some are distractions: Search for baseball fields on the ground below; look out the window at other planes, and remind myself that we’re moving just as gracefully as they are. And some are on the spectrum between weird and pathological: Rock back and forth in my seat like Leo Mazzone, blending the plane’s motion with my own, camouflaging the former in the latter; start the stopwatch on my phone and, without looking at it, try to stop it after exactly one minute, over and over and over again.7
Most of all, I try to convince myself that all of this is ridiculous. I know the math and history—the odds of a crash are infinitesimal; the odds of turbulence tossing a plane out of the sky are even lower. I know that we’ll be ok; that we’ll land safely and were always going to land safely; that planes fly through much worse (by choice, even); that the pursuit of smooth air is about comfort and not safety; that my distress is no more sensible than that of a cat in the car, afraid of something it doesn’t understand.
No matter. Despite putting more faith in data than most,8 actuarial tables are a useless antidote. I can’t bend my feelings around facts; I can’t forge them between a hammer of reason and the anvil of reality. I am at their mercy, captive to them just as they’re captive to every jolt and shiver. My emotions can’t be outwitted, only sterilized—among all of my in-flight therapies, the only reliable one is tranquilizing myself with a self-prescribed dose of borrowed pills.
“If one engine fails, how far do you think the other one will take us?”
“All the way to the scene of the crash.”
When ChatGPT launched out a few months ago,9 my first thought was that we aren’t smart enough to handle what was coming next. Because ChatGPT and other LLM-based AIs can make wrong-but-seemingly-feasible claims about anything, the “marketplace of ideas”—tenuous as that concept already is—would soon be inundated with millions of counterfeit debate club arguments that have no foundational legitimacy. We’d lose track of what’s true, and slowly get lost in Sydney’s sauce.
I’ve since changed my mind. As fact finding tools, AI chatbots have tripped so publicly and so loudly that I suspect we’ll be overly skeptical of what they tell us, just as we were taught to “never trust what you find on the internet” in the early days of the World Wide Web.
However, like Ben Thompson pointed out, chatbots are quickly proving to be astonishingly capable around another domain: Our emotions. That’s what makes today’s chatbots feel so much more eerie than every version that came before. Those were stilted and sterile, like an uncomfortably formal support agent who’s required to read from an unnatural script. But even in these early days, ChatGPT and its siblings are fluid and anthropomorphic; they can, as Thompson says, “make the human it is interacting with feel something.” They are, for lack of a better term, emotionally manipulative.10
What’s more, they seem to do this on their own, in response to users’ prompts and not their own programming. Imagine if they were tuned to tug in this direction, to incite anger, engender fear, promote happiness, or incept jealousy. Imagine if the bots didn’t follow us into our emotional pits, but instead pulled us there.
Though they aren’t marketed as such, AI applications are already being built to do exactly this. For example, consider Tome, a generative storytelling app that’s on a mission to “help anyone tell a compelling story.” Though they claim that this is to enable society to “act on its best ideas,” this justification is self-contradictory. It admits that ideas are chosen not by their substance, but by the stories that argue on their behalf. While good ideas may make better stories, Tome can also make bad ideas look plenty polished too.
In other words, the entire point of Tome, and dozens of other apps like it, is emotional capture.11 And if we can’t resist generic SPCA commercials and hokey political ads, do we really have any hope against the future bots that can take this a step further, and triangulate around our exact aspirations and insecurities? What if every email is generated to make us, individually, feel a very precise way? Every social media site is overrun by bots praising and trolling your posts? Every text message is edited by an assistant that’s dialed, like a host in Westworld, to be funny, flirty, or spiritual? If this is where we arrive, it’s hard to see who’s controlling who—are we in charge of the AI, or is the AI in charge of us?12
Maybe I’m underestimating our collective resistance to such emotional manipulation. But I have no faith in my own ability to face it. As flight after labored flight has shown me, I can’t immunize myself to what I feel. I can’t tell myself that hallucinations are coincidences that mean nothing, like an ominously timed fortune cookie. I can’t resist the machine pushing my buttons by reminding myself it’s just a machine programmed to push my buttons. The artificiality of AI won’t matter any more than data and physics matter to me on a flight plowing through choppy air—bump me with the right magic words, and I’ll be triggered, overwhelmed, my brain on a loop, trying to count to ten.
Not you, perhaps. But that may not matter. All of us are on the same societal plane. If enough of us get blinded by some kind of engineered outrage, we could all be headed to the scene of the same crash.
In another echo from the past, he’s one of the best because he draws inspiration from the movies he watched growing up.
A shower thought about planes: It’s weird that they go all over the world, but only touch tiny parts of it. Like, if a plane was nearsighted, it’d travel everywhere, but all it’d ever see is runways and open sky.
Alaska 82? Nothing to worry about. Delta 321? Iffy. United 990? Lifetime movie.
To be clear, this was surely fine and necessary, even if the proximate cause was a bit senseless.
It was always about the view but never just about the view.
Superior, they said, never gives up her dead
When the gales of November come early.
Artifacts of my fear are scattered across my phone. There are short stories about airplanes vanishing into the breach, the name I gave to the interval of a flight’s climb when it roils through low, dense clouds, and all you can see out of the window is a gray haze cut by blinking navigation lights, like a silent fire alarm flashing in a smoke-filled room. There’s an outline for a novel about new pills that let you sleepwalk through hours and days and years of life, living them but not experiencing them. There are notes about movies that capture planes in extreme slow motion, documenting how violently their engines tear through the air. And there are long lists of numbers—turned into bizarre analyses—bouncing between 40 and 80, recording my guesses of how long a minute actually is.
It’s with great self-loathing that I have to admit that all of this was a lead-in to talk about AI. Or, it’s with great pride that I can say I figured out a way to write about airplanes and stuff while pretending to talk about AI.
I think it’s telling that, thus far, AI’s most captivating application has been simple text generation. Tools like Shazam or image recognition software are in some ways more technologically inexplicable, but they feel computational and encyclopedic. Conversation, by contrast, is expressive, affecting, human.
To be fair to Tome, their version of this is mostly benign; it seems that they primarily help people write better fundraising pitch decks. Which, if nothing else, answers this question.
I don’t mean that the AI is sentient. It’s more akin to an addictive drug, which can control people without being alive, or self-aware, or anything more than an inert pile of powder.
I know all of these exact behaviours, I used to incredibly afraid of flying, but I never wanted it to prevent me from traveling. So all of the counting, listening to changes, obsessively watching the little plane on the map as we progress to finally be done with the terror tube. Being a data person, I decided that it was a "good idea" to consult the NTSB accident and incident reports (https://www.ntsb.gov/Pages/AviationQuery.aspx) which sort of helped, because there you see the majority of incidents are your garden variety "whoops!" on the tarmac between vehicles, severe turbulence that injures someone who doesn't pay attention to the seatbelt sign, or some problem with the APU. (not life threatening) Then one day, my fear just stopped. I don't know how, I was doing a lot of flying at the time, so maybe that was it, but now I don't have that gut twisting dread whilst flying. (Save for a few lingering tics)
But to your point about AI and emotional manipulation, I think this is a very real thing that needs to be explored further. I suspect there's some very smart AI Ethics people that are on it, but not getting the attention that it needs.
On the flying anxiety part. For me, visiting a cognitive behavior therapist helped a lot. More specifically - understanding distinction between rumination and "productive" thought. And now, when I sense the begining of a "negative loop" ("counting down the long minutes until we land"; "remind myself the wings are still upright"; "I monitor for changes in pressure or speed, listen for new engine pitches or hisses in the ventilation, and try to detect hidden connotations in announcements") I'm trying to shift me focus onto something more "productive" (listen to podcasts or music; read a book; watch a movie; sleep).
On the "are we in charge of the AI, or is the AI in charge of us?". As with anxiety, the goal is not try to control anxiety (the more I tried, the more it took control of me), but accept it and let it be (it usually goes away in 20 minutes without additional negative food, as our brain not designed to stay in this state for too long without substantial reason).