Democracy is dead.

Not necessarily the concept of democracy. “Rule by the people” is a hard meme to kill. But the seventeenth-century liberal democratic nation-state is stumbling, wounded, and stupefied. Its murderer: technocracy. Technocratic control systems have faster OODA loops. They can iterate faster, test hypotheses rigorously, spread virally. They ran circles around democracatic governments, gouging and carving at the tissues of bureaucracy until the civic life drained from the body politic.

The 2016 American election got a lot of people woke to the technocracy, but it wasn’t an isolated phenom. Cambridge Analtick and Faceboor and Russian nesting trolls are all symbiotic parts of the same ecosystem. They’re behavior designers, influence marketers, choice architects: dark wizards.

Dark wizards have always been a threat to status quo powers. Remember Rasputin? People who se the emergent opportunities of systems are better situated to take advantage of them – and people with no ethics take less time to consider the consequences of their actions. Scumbags get the first-mover advantage.

The exploit we’ve created now is a direct, constant, and instantaneous control over human motivation . We’ve built these exobrains, these extensions of our neural circuitry, and opened the doors wide open for anyone with an internet connection to stroll right in.

The smartphone is an addiction machine. It’s always on, always connected, and always nearby. It’s got all the juiciest bells and whistles that Pavlov could have wished for: sounds, lights, colors, motion, push notifications. And don’t get me started on ringtones that sound like a person whistling for their dog. Have some dignity, people.

All these hyperstimuli wire directly into your limbic system. Dopamine and cortisol, desire and suffering, are wrenched from their delecate perch and made to dance to the whims of mad King Capital. How can slow, deliberative democracy stand a chance?

The limbic hack is still rudimentary, of course. It’s not the totalitarian mind-control machine that your electro-billionaire might hope for. No, we’ll have to wait a while for neural mesh and dream advertisement and all the “fun” cyberpunk stuff in R&D. But even a dull scalpel can be used for brain surgery, if you don’t care too much about your victim. I mean, patient.


To show you how easy it is to destroy a democracy using modern technology, let’s do a little “behavior design”. I’ve got a notion of a “killer app” for propaganda: the fanboi botnet. Who needs an army of Russian hackers, when there are millions of phone addicts who will work for free? That is, if we program them right.

######

NOTE: I haven’t searched to see if this is a real app or not. For the sake of the experiment, I’ll imagine this from scratch, but I wouldn’t be surprised at all to find it already exists. If you know of it, drop me a line at max@permafuture.net

Say you’re a propaganda outlet, a fake news site or a political blog. You make all this content, well-written essays that could persuade any audience. Only, the competition for attention on the internet is unrelenting. You have to shoulder your way through millions of competing messages to get any audience at all. But as we all know, the best way to get read is to be read, because social media relies on network effects. It’s word of mouth on steriods.

The goal is simple: get an army of addicts to “share” and “like” your propaganda on all social media outlets.

Instead of hacking a million IoT devices, we’ll spin up a botnet of humans. This is a secular, nonpartisan project. It doesn’t matter whether we exploit the weak-minded ideologues of the other party, or the highly-networked geniuses of our team. With the same principles used by the most popular apps in the world, we can do this to anybody.

We’ll use the technique provided by the questionably-ethical, suspiciously well-dressed  at Boundless.ai: the CAR model. CAR stands for Cue-Action-Reward , the essential feedback loop of a habit (or an addiction). We want our fanbois to  develop a habit of sharing our content, not just once, but on every social media account they have.

CAR model diagram

We know the Action we want – click the “share” or “retweet” button. We want that to happen repeatedly. Not just once for each story in your feed, but once for each social app, for each story. So the atomic form of the action is just that: click a share button. We’ll program the habit so it repeats by itself.

We still need a Cue, and a variable Reward. The cue can be as simple as the row of branded icons at the side of every article on the net. But users are immunized by having seen so many of these. Every time the user sees the cue and doesn’t do the action, the habit is weakened. So we’ll make ours special.

Let’s make the icons react to whether or not they’ve been shared by our users . Rather than the little numbers you sometimes see next to these buttons, showing how many times they’ve been shared on those sites, we’ll only count the shares from people who are logged in to our propaganda outlet. We want users to be motivated to add their shares and likes to the least loved articles, so we’ll keep the brightly colored icons that the Socials give us. But as more of our users share an article, the icons will lose their luster, dimming and going gray. If all of the registered users have clicked a particular button, the icon will suddenly turn solid gold to represent a “win”.

We now have a cue: if an article has a colorful icon, click it and “share” the page. We also have a reward: once you’ve shared on a particular service, the icon gets a little bit grayer. And we’ve made the reward variable (remember, variable rewards cause a much stronger dopamine effect than consistent rewards): the amount that your share affects the color change is very small, unless you’re the last one to push the button. Then you get the gold. And as long as we don’t tell you how many users have yet to click it, you’ll never be sure which effect you’re going to get. Instant addiction.

But if we want to have an effect on the scale of nations, we’re going to need a more powerful hook than simple color changes. Let’s take it one step further.

Humans are social creatures. We have deep needs for social connection and recognition. Those needs can be hacked.

Let’s add a bit of “gamification” to our propaganda machine. We’ll make it social, and fun, and leverage the human urge to achieve greatness in the eyes of their peers.

Gamification has three attributes: points , badges , and leaderboards .

Points are easy. We’ve already focused on a metric: amount of articles shared, times amount of apps shared on. Let’s make the (seemingly innocuous, and technically simple) design choice to display that number as flair next to each user’s avatar. Wherever they comment on the site, the number will sit right next to their name and picture, crowning or condemning them in the eyes of their peers. We now have a social motivation for sharing, much more powerful than the simple color-change reward.

Badges are earned titles, like Scouts earn, or like adding “Sir” to the name “Ringo Starr”. Let’s give each user a gold star for every time they “won the gold” on sharing buttons. To mix things up a bit, and encourage people to share early and widely, let’s give a platinum star for users who are the first to share an article on a given site. Since we want these shares to rack up quickly, we’ll put a little counter for each, right there next to the counted shares. Your prowess as a share-clicker is now displayed to all, so your desire to be first and/or last goes up, and your addiction gets stronger with the social reinforcement that other people are competing for the same badges.

Leaderboards give a sense of social context to the points and badges. Like the Top Scores on an arcade game, leaderboards directly display the rank of all players in the social game. Humans have a deep desire to know their social standing, and very little explicit evidence to work from. So if we give our little trolldolls a visible metric from which to judge themselves, they’ll compete against each other to do our bidding the best. A simple list of users, organized by most points, would suffice, but let’s add some juice to keep it interesting. We’ll put three big splash pictures at the top of the chart, showing the users who have the most Shares, Gold Stars, and Platinum Stars, respectively. We can give them special titles and cute little cartoons, or allow them to display a motto or favorite quote. Just creating this special Position of Honor motivates anyone who’s even near the top of the ranks to give it just that little bit more – and this is a feedback loop too, as each new face in the place of honor expands the special social club of those who care waaaay too much.

Now we’ve added peer pressure, quantified status and in-group/out-group dynamics to our simple dopamine habit-loop. We’ve also created an incentive for users to sign up with our site: the share buttons might change colors for you as a lurker, but you’re throwing away all the points you could be collecting and the trophies you could be showing off! So we’ve got a dedicated group of readers who will share every article we post, on every channel, from the moment it’s posted until every last one of them has run out of places to share.

We have a fanboi botnet.


This idea wasn’t hard to come up with, and it wouldn’t be hard to code. It’s a terrifyingly simple design. Any fringe news outlet could implement this right now and become the next Breitbarf or HuffPose. I’m tempted to make it myself, just so a reasonable person would be able to pull the plug when it inevitably goes horribly wrong. We’re just beginning to understand human neurology, and mind control is still a blunt instrument.

But no matter how strong your political institutions are, a blunt instrument to the head is still a blunt instrument to the head. Behavior design is powerful enough to give democracy a concussion. Will it be enough to kill?

As always, thanks for reading.

– Max


SCIOPS is a weekly newsletter about cognitive security. Feel free to forward it to anyone you think would like it, or share it on your social-tracking profile. If you have thoughts, questions, or criticism, just respond to this email.

If you’re seeing this for the first time, make sure to sign up at tinyletter.com/sciops for more cyberpunk weirdness in your inbox every week.

You can read previous issues of SCIOPS at tinyletter.com/sciops/archive