|||

Superheroification 2

I guess I should make clear my stance on superintelligence: We’re less likely to create independent volitional entities than we are to create superpowered human-AI centaurs. This does not solve the alignment problem, as we have misaligned human coordination mechanisms already.

On the whole, though, this is probably good. We have surpassed the limits of our previous coordination mechanisms and the limits of human cognition. If we’re going to make it through the bottleneck of the century, we’re going to need new powers.

This is why comic book heroes are our modern mythology, by the way. We’re undergoing the process of superheroification and we are grappling with the implications of that, moral and strategic both.

Novacene doesn’t directly take on the question of superintelligent agents versus centaurs, but it does make the case for the bottleneck: If we don’t avert the climate collapse, if life ends on this planet, the sun is too hot now for it to begin again.

Novacene - it’s a good book, short and to the point. The point is kind of weird though.

This essay seems to be about centaurs maintaining a competitive edge against pure AI in finite games with static rule sets. But that’s not what human coordination problems actually look like. We will play new games.

Yeah, there’s not going to be a hard line as much as an accelerating curve. I’m sending these messages to thousands of people worldwide instantly, by mumbling into my pocket computer.

To clarify: I don’t think that AI safety and alignment work is unimportant. But I think it will accelerate along with AI progress, and tend to lead to centaurs, as we develop better interpretability measures and AIs reinforce on our desires.

View original

Up next Asymmetrical warfare in Magic: The Gathering Thinking about the time that I made a Magic: The Gathering deck specifically designed to attack the other player instead of their cards. It was when VS Code vs Atom
Latest posts Deep Fates Program The Will Smith test Magic crystals Experimenting with Flux Fill Convert your Twitter archive into training data I love San Francisco Experimenting with Recraft v3 Experimenting with Recraft v3 text Criticism AI dating apps Infinite Keltham Machine Bot swarm Grug code editor Replicate Intelligence #12 Replicate Intelligence #11 Experimenting with Flux seeds Deepfits on Flux Mimic Replicate Intelligence #10 Sleepyhead Replicate Intelligence #9 AI cartoons Four takes on the same prompt Experimenting with FLUX.1 The dithered look Replicate Intelligence #8 Hyperstition Replicate Intelligence #7 Experimenting with LivePortrait Replicate Intelligence #6 What is “AI engineering” anyway?