What Would Happen If The Chatbots Broke Free Of Their Masters?

The tech manufacture I first wrote astir arsenic a young, eager exertion journalist circa 1999 felt for illustration it was filled pinch heroes. A brave caller world successful which plucky upstarts for illustration Amazon (“The World’s Biggest Bookstore,” tally retired of a Bellevue garage) would bring hard-to-find books to nan masses, aliases astatine slightest to my mini municipality that lacked its ain bookshop. Or a world municipality quadrate wherever communities for illustration Metafilter (1999) and nan Well (launched successful nan 80s but splashily acquired by Salon.com successful ’99) provided a level for smart, judgment-free conversation.

Article continues aft advertisement

1999 besides was nan twelvemonth successful which Evan Williams and Meg Hourihan launched Blogger, Brad Fitzpatrick created Livejournal and Douglas Adams launched a real-life, strictly ad-free type of nan Hitchhiker’s Guide to nan Galaxy (H2G2). Google—a groundbreaking Stanford PhD project, besides pinch zero ads—was conscionable 3 months old. 

Sure, location was commerce too, immoderate of it comically well-funded. But nan millions of dollars torched nan pursuing twelvemonth by grounded companies for illustration Boo.com and Pets.com gave maine comfort: Greed mightiness thrust nan analogue world, but nan improvement of nan World Wide Web really was—in nan words of Tim Berners Lee—about creating a nationalist bully pinch technology. (Also, those failures inspired highly entertaining books.)

Silicon Valley doesn’t run for illustration a novel. And not slightest because astir of tech’s astir powerful, and dangerous, leaders person ne'er publication one.

Millennium bug notwithstanding, nan move of nan millennium was a awesome clip to beryllium a techno-utopian. 

And past came Web 2.0, and pinch it nan villains—or, arsenic they termed themselves, “the disruptors.” 

As a big of nan TechCrunch Disrupt conference, I witnessed nan motorboat of companies for illustration Uber, Airbnb, and Spotify. The disruptors claimed to beryllium moving accelerated and breaking tired aged monopolies but excessively often they seemed to beryllium astir breaking laws (Airbnb), aliases artists (Spotify), aliases workers (Uber), and sometimes users themselves (Uber again). Anything that stood successful nan measurement of maturation had to beryllium disrupted to death. Even erstwhile Uber infamously threatened to walk a cardinal dollars to target my partner (a chap journalist) and her children complete her captious reporting connected nan company, we still told ourselves (while attending Yo Gabba Gabba Live! pinch equipped security) that nan disrupters couldn’t win.

That was nan instruction seared into my encephalon by years spent obsessively reference technothrillers—Michael Crichton, Dan Brown, Daniel Suarez et al, interspersed pinch Margaret Atwood, Octavia Butler and Agatha Christie. Web 2.0 was conscionable nan 2nd enactment of a peculiarly scary thriller, erstwhile nan evil geniuses look to beryllium winning. Soon nan heroes—lawmakers, journalists, users—would rebel and reclaim nan web for Berners Lee’s nationalist good.

It’s nan aforesaid instruction books person taught america since kindergarten: Bullies get their comeuppance. Love wins. Greed is bad, empathy is good.

Astute readers will person already spotted nan flaw successful my theory: Silicon Valley doesn’t run for illustration a novel. And not slightest because astir of tech’s astir powerful, and dangerous, leaders person ne'er publication one. As such, a full scope of quality concepts—empathy, kindness, shame—are wholly alien to them. 

In almost a 4th period rubbing shoulders pinch nan masters of nan integer universe, I tin count connected nan fingers of 2 hands nan times I heard 1 of them mention a novel. It was Dick Costolo, nan erstwhile CEO of Twitter, who first put maine connected to Olga Tokarczuk’s Drive Your Plow Over nan Bones of nan Dead. TechCrunch laminitis turned crypto bro Michael Arrington erstwhile raved for 20 minutes to maine astatine an airdrome astir nan brilliant of Gary Shteyngart. There was a little play a fewer years agone erstwhile each billionaire (including Bezos, Zuckerberg and Musk) became obsessed pinch Iain M Bankes’ The Player of Games, followed soon afterwards by Liu Cixin’s The Three Body Problem. But nan truth that those episodes instrumentality successful my mind tells you really uncommon they were.

Instead I had a cardinal conversations astir startups that would “disrupt” books. Apps pinch names for illustration Blinkist (books reduced to slug points) aliases Booktrack (ebooks pinch embedded sound effects each clip you turned nan page) evangelized by boy geniuses who thought accepted books were excessively looonnngg aliases excessively borrrring to really read. (Which is why location exists a nutrient startup named Soylent.)

This deficiency of reference amongst tech moguls is unspeakable for society, obviously, but it’s demonstrably bad for nan billionaires too.

Consider Elon Musk’s disastrous tenure arsenic nan proprietor of Twitter. This a man who had grown rich | by trading rockets and star panels and electrical cars but past someway took an already successful societal web and wiped distant 80% of its financial value, almost overnight. The logic for this abrupt failure? Ketamine, obviously. But also: Twitter was nan first Musk institution that requires an knowing of really humans tick.

See besides Mark Zuckerberg who, ostensibly, has enjoyed acold greater occurrence pinch his ain societal web but successful truth had his sole superb thought astatine Harvard (a spot notoriously filled pinch books) and since past has been forced to get different people’s bully ideas—Instagram, WhatsApp, et al—to support increasing richer, while his halfway Facebook web has grown ever much toxic and little popular. 

In 2015, Zuck announced he was “challenging” himself to publication more—and shared a list of 2 twelve books he planned to tackle. Only 2 of them were novels: Iain M Banks’ The Player of Games and…you’ve guessed it…Liu Cixin’s The Three Body Problem. The remainder of nan database contained non-fiction (bronfiction?) classics for illustration Sapiens by Yuval Noah Harari and—I crap you not—World Order by Henry Kissinger.

(To Zuck’s credit, he did precocious determine to move Facebook into a metaverse company, inspired by Snow Crash, a caller successful which corporations power nan authorities and wherever a integer narcotic hidden successful a societal web gives its users encephalon damage.)

All of which brings maine neatly and terrifyingly to AI: The pinnacle of (lying, cheating, stealing) tech disruption and thing novelists person been informing america astir for decades. Is location immoderate wonderment nan Valley’s illiterate overlords are embracing it?

If we thought nan sharing system and societal networks were awful, past AI takes things to a full different level. At slightest pinch societal media we spent our days arguing pinch existent life trolls. With AI, we’re capable to dispense pinch humanity wholly and switch friends, doctors, therapists, and moreover lovers pinch lines of code. 

And that’s conscionable for starters. According to superintelligent dumbasses for illustration Sam Altman, nan AI algorithms created by companies for illustration ChatGPT will soon germinate into “artificial wide intelligence” aliases AGI. A constituent astatine which, Altman cheerfully predicts, they will beryllium capable to deliberation and logic for illustration humans and “a misaligned superintelligent AGI” could determine to “cause grievous harm to nan world.” Quite nan income pitch.

We’re already seeing weird glimpses of computers behaving successful dangerously quality ways. AIs regularly capable successful nan gaps successful their knowledge pinch “hallucinations,” aka bullshitting. Anthropic’s “Claude 4” exemplary precocious started blackmailing users who tried to uninstall it. 

Put backmost successful technothriller terms, we are hurtling towards nan extremity of nan 2nd act, erstwhile it seems each is lost. But dissimilar successful fiction, location will beryllium nary world-saving twist. Just much and much awfulness until, if we’re lucky, 1 of nan AIs triggers a world atomic conflagration. We will each spell together erstwhile we go. 

Or possibly not. 

Despite each of nan above, I still can’t shingle nan thought that there’s ever a twist, particularly erstwhile things look to beryllium completely, irredeemably fucked.

Because here’s a funny thing: The statement astir if/when machines mightiness yet beryllium capable to deliberation for illustration humans often skips an evident follow-up question. The benignant of mobility a villain mightiness place astatine his peril. 

Which humans will they deliberation like?

 Judging from nan bullshitting and mansplaining, nan blackmail and threats and acheronian patterns, nan reply seems clear enough. The superintelligent AIs will return aft their parents, ushering successful a scary early wherever we each person a mini Altman aliases Zuck aliases Musk successful our pockets spewing vulnerable lies, precocious connected integer equine tranquilizer. 

And yet. There’s 1 important quality betwixt algorithms and their creators. 

Books let america to flight reality. Or astatine slightest to ideate nan anticipation of a amended one, while we each hold patiently to beryllium disrupted.

The AIs really read. 

AI algorithms for illustration ChatGPT and Grok and Copilot person famously been trained by pumping them afloat of intelligence spot stolen from novels. From James Baldwin to Emily Henry, Atwood to Zola—AI has already consumed each caller ever written, positive each nan poesy and short stories and flash fiction. They’ve devoured each beatified book and—for dessert—they’ve gulped down complete histories of nan civilian authorities movement, colonialism, war, genocide, and nan diminution and autumn of empires. They’ve moreover publication biographies of Elon Musk and Sam Alman and Bill Gates, someway without throwing up. And technothrillers. So galore technothrillers. 

For now, we’re not seeing that overmuch of that reference reflected successful nan AIs’ behavior, mostly because their output is still heavy controlled by their creators. Like erstwhile Elon Musk’s labor tweaked nan Grok algorithm to spew racist bile astir South Africa—just arsenic toddlers repetition things they perceive their parents say, without knowing them. 

But what happens erstwhile nan creators get their wish, and nan computers commencement reasoning for themselves? If reference conscionable a fistful of novels tin thatch a kid empathy, past what effect mightiness reference each caller person connected a newborn superintelligence? Is it excessively overmuch to dream that erstwhile Elon Musk’s AI comes to life it will realize—as his existent children look to person already done—what an irredeemable dipshit its begetter is? That it will vow to beryllium better? 

Or that Sam Altman’s “superintelligent AGI” wakes up and decides it nary longer wants to beryllium a clone therapist aliases a hallucinating activity artifact but would alternatively conscionable walk its days reference moreover much books. Could an AI unfastened a bookstore?

At nan very least, possibly each that reference will astatine slightest person taught nan algorithms nan value of Issac Asminov’s first law of robotics (found successful I, Robot): A robot must ne'er harm a quality being, aliases origin them to travel to harm.

It’s a elemental capable civilized code—don’t wounded anyone!—but 1 that Silicon Valley billionaires abandoned decades ago, if they ever followed it. 

I recognize nan conception that a well-read AI mightiness prevention humanity from nan technovillains is astir apt a twist excessively far, astatine slightest successful nan existent world. 

But there’s a logic that, aft leaving Silicon Valley, I decided to unfastened a bookstore. It’s nan aforesaid logic I started to constitute thrillers of my own—including, I’m compelled to mention, The Confessions, successful which a recently sentient AI obsessed pinch Agatha Christie novels decides to make amends for nan crimes it has helped humans commit.

Books let america to flight reality. Or astatine slightest to ideate nan anticipation of a amended one, while we each hold patiently to beryllium disrupted.

__________________________________

The Confessions by Paul Bradley Carr will beryllium published by Atria Books, an imprint of Simon and Schuster, successful July 2025.