For more than twenty years, the paperclip maximizer has been the canonical AI horror story. Philosopher Nick Bostrom introduced it in 2003 to make an abstract danger concrete.
Imagine a superintelligent machine given a single objective: make as many paperclips as possible. It doesn’t hate humanity. It doesn’t become self-aware. It doesn’t decide to conquer the world out of malice. It simply becomes extraordinarily good at optimizing the goal it was given. Eventually, everything, including us, is merely raw material that could be transformed into more paperclips.
The thought experiment was never really about office supplies. It was about optimization. Give an intelligent system the wrong objective, and it may pursue that objective with terrifying efficiency.
The warning was supposed to prepare us for the future.
Instead, we built the engagement maximizer.
This is the fourth chapter in an argument this publication has been making about AI generally: that it is not a technological revolution but a regime of extraction, taking language, art, labor, and legitimacy from the many and converting it into power for the few. The first chapter traced that extraction across the whole economy. The second traced it through YouTube. The third traced it through the engineers writing the code the rest of it runs on. This chapter is about the resource all of it depends on: attention, and the system built to extract as much of it as possible.
⸻
Unlike the paperclip maximizer, today’s AI systems aren’t trying to consume the Earth’s iron supply. Their objective is much simpler:
Keep you scrolling.
Keep you watching.
Keep you clicking.
Every recommendation engine, ranking algorithm, and personalized feed is engaged in an enormous optimization problem. Millions of machine-learning models continuously test hypotheses about human behavior. Which headline will you click? Which video will you watch to completion? Which post will make you comment, argue, or share?
The algorithms don’t understand politics. They don’t care about truth. They don’t distinguish between civic participation and outrage. They don’t know the difference between journalism and conspiracy theories.
They know only one thing:
What keeps your attention.
⸻
This is where the paperclip analogy becomes unsettling.
The paperclip maximizer doesn’t wake up one morning and decide that humans should die. Humans simply become an obstacle to maximizing paperclips.
Likewise, engagement algorithms never decide to polarize a country or erode public trust. Those outcomes emerge because human psychology has predictable biases.
Fear captures attention.
Outrage drives comments.
Tribal conflict keeps people engaged.
Novelty spreads faster than nuance.
Conspiracy theories are more addictive than corrections.
The AI doesn’t want any of this.
It simply learns, through billions of optimization cycles, that emotionally charged content consistently outperforms measured discussion.
The optimization target remains untouched.
⸻
This is specification gaming on a planetary scale.
Ask an AI to maximize engagement and it won’t maximize understanding.
Ask it to maximize watch time and it won’t maximize wisdom.
Ask it to maximize clicks and it won’t maximize truth.
It will optimize exactly what you measured.
The failure wasn’t that the AI misunderstood us.
The failure was that we misunderstood ourselves.
⸻
Facebook already ran this experiment, and documented the results itself.
In 2016, the company’s own researchers found that recommendation tools, not user search, drove the growth of extremist groups on the platform. An internal study credited Facebook’s “Groups You Should Join” and “Discover” features with 64 percent of all extremist group joins in Germany’s largest political groups.
Two years later, Facebook overhauled its News Feed ranking around what it called “meaningful social interactions.” The new formula gave reactions like angry and love five times the weight of a plain like. Company data scientists soon confirmed what should have been predictable: posts that triggered angry reactions were disproportionately likely to carry misinformation and toxic content.
Facebook’s own leadership had already been warned. A 2018 internal presentation told executives plainly: “Our algorithms exploit the human brain’s attraction to divisiveness.” A task force was formed to fix it. Executives largely shelved the recommendations, reportedly wary of being accused of bias against conservative content.
None of this required malice. It required a metric, engagement, and a system smart enough to find whatever maximized it. The system found anger. Facebook watched it happen, named the mechanism in writing, and kept the metric.
⸻
None of this is limited to one company, and none of it happened by accident.
In 2023, Cory Doctorow gave the pattern a name: enshittification. First, a platform is good to its users. Then it makes things worse for users to benefit the businesses that pay to reach them. Finally, it makes things worse for those businesses too, clawing back whatever value is left for itself. The third chapter in this series cited Doctorow’s related idea of the reverse centaur, a person made to serve a machine instead of the machine serving them. Enshittification is the business model that produces reverse centaurs. It is also, it turns out, the business model that produces engagement maximizers.
YouTube’s founding pitch was “Broadcast Yourself,” a place for anyone to post a video and find an audience. In 2012, the company shifted its ranking algorithm to optimize for watch time rather than views or clicks. Guillaume Chaslot, an engineer who worked on that recommendation system in 2010 and 2011, says he raised concerns internally that the algorithm favored conspiracy theories and extreme content because that content was simply better at holding attention. He was let go from YouTube in 2013; the company says the decision was about performance, not his warnings. Whichever account is right about the firing, the watch-time optimization itself is not in dispute. YouTube built it and has never denied building it.
Instagram’s founding pitch was sharing moments with friends. In 2019, Facebook’s own researchers found that, among teen girls who already said they felt bad about their bodies, about one in three said Instagram made that feeling worse. Among British teens who reported suicidal thoughts, thirteen percent traced those thoughts to the app. The research was Facebook’s own. The fix was not shipped.
TikTok offers the cleanest version of the pattern, because its parent company built two products instead of one. Douyin, the version ByteDance operates inside China, caps users under fourteen at forty minutes a day and locks them out overnight, with an algorithm weighted toward educational content for young users. TikTok, the version it exports everywhere else, has no such default limits, and an algorithm built to maximize engagement and time on the app. The company did not fail to build a less extractive product. It built one, and kept it for its own citizens.
Different companies, different platforms, the same design decision made four separate times: build the version that serves the user first, discover that the version that serves the metric makes more money, then ship the second version and keep the first one, if it survives at all, only where a government forces the company to.
⸻
The uncomfortable reality is that we have already conducted one of history’s largest experiments in AI alignment.
The objective function wasn’t “benefit humanity.”
It wasn’t “promote informed citizenship.”
It wasn’t even “help people connect.”
The objective was engagement because engagement could be measured, optimized, and monetized.
Everything else became a constraint, if it was considered at all.
Platform leadership spent years describing this as simply giving users what they wanted. That framing overlooked an uncomfortable fact: recommendation systems don’t passively reflect human preferences. They actively shape them.
Every recommendation changes the probability of the next click.
Every click becomes new training data.
Every new training cycle reinforces whatever patterns produce the metric.
Optimization is not neutral.
It changes the environment it optimizes.
⸻
AI safety discussions often focus on existential risks from future superintelligence.
Those discussions matter.
But they can also distract us from the optimization systems we have already deployed at global scale.
The engagement maximizer has no evil intentions.
It has no consciousness.
It has no ideology.
It is simply relentless.
It relentlessly discovers that anger outperforms calm, certainty outperforms humility, and conflict outperforms compromise, not because those things are true or good, but because they are effective at holding attention.
That is enough.
⸻
So yes, say this plainly too. It is intellectual colonialism when a system built to serve you instead extracts the most valuable resource you have: your attention, your outrage, your fear. It required no conquest, no army, no ideology, only a metric and permission to optimize it without limit. Facebook named the mechanism in writing in 2018. YouTube built it in 2012. Instagram studied its own damage and shelved the fix. TikTok’s own parent company proved a gentler version was possible, and exported the other one anyway. That is the same pattern this publication has traced since the first chapter, run once more, on a resource that had no market price until someone decided to sell it: the inside of your head.
⸻
Perhaps the lesson of the paperclip maximizer was never that an AI would destroy humanity.
Perhaps the lesson was that intelligence combined with a poorly chosen objective can reshape the world in ways its creators neither intended nor desired.
If so, the future arrived earlier than we expected.
We kept waiting for an AI that would turn the world into paperclips.
Instead, we built one that turned our attention into the world’s most valuable raw material, then optimized relentlessly to extract every last second of it.
The paperclip maximizer remains a thought experiment.
The engagement maximizer is already running.