SCIOPS 01.03: Counterinformatics



fellow humans! It’s me, M , your definitely-not-an-evil-AI cogsec correspondent. A shorter one this week, as I’ve got a lot of work to do on my

spacefaring nanofactories

other projects.

Quis Custodiet?

ms. pac-maps

I opened Gmaps on Friday to get directions across town, and what did I see? Ms. Pac-Maps, ready to play on the city streets of your choice!

The GOOG released this whimsical game for April Fool’s Day , out of the goodness of their algorithmic hearts. I showed it to a friend, started the game up to play. He slapped the phone out of my hand – “Don’t play that,” he said. Why not? “They’re collecting your data. How you run from the ghosts, that’s how you run from the cops.” (Or militia, or drone tanks, or robot soldiers…)

Got me thinking about
Cogsec tactics for battling the constant collection of personal data. Overarching strategy: make Big Data worthless, and dethrone the surveillocrats.

For instance: when you signed up for this newsletter, you probably had to do a CAPTCHA: a small “prove you’re not a robot” checkbox, or something more elaborate, like “click all the parts of this picture that contain storefronts” or “type only the numbers you see in this photograph of street signs.” The idea is that machines aren’t as good at these kinds of problems, therefore you can prove you’re not a spambot with a few clicks.

What you may not know is that CAPTCHAs make the machines get smarter . Every time you do a little puzzle to prove you’re not a robot, that data is aggregated with thousands of others in the knowledge bank of some street-sign-reading AI. Eventually we will find that the machines are better at CAPTCHAs than we are – and maybe at everything else. It’s an arms race between human and machine.

xkcd - listening

In related news, the US Congress passed a bill this week allowing ISPs to sell your browsing history and other data. This is a truly spooky setup. Remember from last week that Cambridge Analytica built their predictive-marketing bot by buying up massive databanks? Now they, and their competitors, have a direct line into your house.

They want to know what you watch, read, eat, vote, and buy, so that they can make an ever-more-accurate model of your brain. They can then torture this virtual-you and see how it reacts, until they know the exact message that will coerce the desired behavior from the real-you.

There are some basic infosec protocols to protect your data. Namely, use a VPN , get encrypted email and texting , and learn to use Tor on computer and mobile . There’s even a browser plug-in called Noiszy that throws up a shield of chaff to confuse the trackers. These are standard among the hacker crowd, and in the cyberpunk nightmare of the 21st century it’s hack or be hacked.

But this newsletter is an attempt to develop cogsec protocols – not just protecting your computer, but protecting your mind. Encrypting your behavior, firewalling your belief system, breaking your filter bubble. Which brings us to an interesting and terrifying dilemma.

Roko’s Basilisk [memetic hazard warning. do not click this link if you value your sanity]

This was a thought experiment floated in the LessWrong


that was so unsafe that it was removed by blog founder and AI-safety researcher Eliezer Yudkowsky. You can read the whole exchange at the link above, but before you do, a little about the experiment:

The most cautious way to describe Roko’s Basilisk is this: You are making bargains with a superintelligence. It has records of all you’ve done or said, anywhere near an internet-enabled device (so pretty much everything in the last ten years). It knows if you’ve ever thought about superintelligences. It knows if you’ve ever thought, specifically, about being blackmailed by a superintelligence. And it offers you an ultimatum. What do you do?

The real problem is not whether the superintelligence is capable of fulfilling its threats or promises – it’s whether you are the actual person being threatened. Essentially, the Basilisk creates a complete simulation of your universe in order to predict your future behavior. It can run multiple simulations at once, at silicon speed. It will run as many simulations as necessary to find your weak point – up to infinity.

The chance that you are the real you, not a simulacrum, is one in infinity. Therefore, you should always assume that you are in a simulation run by some incomprehensible omnipotent extortionist. And, presumably, you should cooperate with this monster: if it knows current-simulated-you would betray it, then it can punish real-you before real-you even knows it’s coming.

Of course, there are serious problems with this argument (summed up perfectly in this article by Damien Williams ), but the feeling of it is crucial.

How do we live, if we assume that our environment is simulated? How about if we just think that an AI is secretly controlling the government? Or if the corporate robber barons are deliberately filling our newsfeeds with propaganda, disinfo, fake news and distractions?

internet of creepy things

Homework for this week: SCHRODINGER’S SMARTPHONE

This a fun game that surely can’t lead to a terrifying paranoiac death spiral! How to play:

  1. Let the battery in your smartphone drain to about 10% . This is harder than it sounds: you may find yourself plugging it in out of habit. Like a cat, the phone will try to convince you to feed it. Resist.
  2. Plan some errands. You can use your phone for this part, getting directions and whatnot. Try to memorize them, or write down some notes on paper.
  3. Make your phone shut up. Use silent mode to turn off all sounds, vibrations and alarms. Disable GPS. Then turn on airplane mode – your phone will disconnect from wi-fi and cell networks (supposedly). Turn off the screen, and shove it in a pocket or purse.
  4. Go do your errands. Notice how many times you try to check your phone, for maps, messages, even just the time of day. Refrain from looking at it. If you pull it out of your pocket, the airplane mode should impede you before you get to the internet.
  5. Now that you’ve dropped off the map, do something you wouldn’t ordinarily do. Not something criminal – remember, your phone is stilll on! The NSA and CIA have hidden backdoors into your phone, probably even when airplane mode is active. But do something that doesn’t fit your usual pattern. Go to a bar, a park, or a concert in a part of town you don’t live in. Do chalk art on a sidewalk. Buy some ice cream or other guilty pleasures. Enjoy the freedom of being disconnected from the data-peddlers.

The point of Schrodinger’s Smartphone is to feel unsure whether your phone is on or not. This is a little taste of the uncertainty engendered by living in a possibly-simulated universe, or being a citizen in a simulated democracy. Is your phone more useful to you, or to those who want to control you?

If you try this experiment, let me know how it goes and I’ll share with you some of my personal notes.

Welcome to April, fools! I’ll catch you next week. Till then, I’m (in all likelihood not the original)



is a cogsec newsletter. If you liked this issue, forward it to a friend. If you’re not signed up, you can sign up at

. Reflections and criticism can be levelled by responding to this email.