Holy shit but that is a bad survey. The bias is real.
It opens reasonably, by asking whether we think AI risk is worth studying -- I agree that it is, because it's a potential extinction-level event if an AI with sufficient power acts against our interests. But then the fucker concludes with a bunch of leading questions which all prejudice participants towards viewing AI as a threat.
Not to mention that the essays, even ones unrelated to AI risk, are pretty shit themselves.
Here's one:
David Chalmers has recently raised the question of whether we live in a computer simulation. You probably heard about this idea long ago, but public discussion of the possibility was recently re-ignited when Elon Musk came out as an advocate.
But a skeptic raised a good point: even simulated civilizations will have the ability to run simulations of their own. But a simulated civilization won’t have access to as much computing power as the one that is simulating it, so the lower-level sims will necessarily have lower resolution. No matter how powerful the top-level civilization might be, there will be a bottom level that doesn’t actually have the ability to run realistic civilizations at all.
This raises a conundrum, I suggest, for the standard simulation argument — i.e. not only the offhand suggestion “maybe we live in a simulation,” but the positive assertion that we probably do. Here is one version of that argument:
1. We can easily imagine creating many simulated civilizations.
2. Things that are that easy to imagine are likely to happen, at least somewhere in the universe.
3. Therefore, there are probably many civilizations being simulated within the lifetime of our universe. Enough that there are many more simulated people than people like us.
4. Likewise, it is easy to imagine that our universe is just one of a large number of universes being simulated by a higher civilization.
5. Given a meta-universe with many observers (perhaps of some specified type), we should assume we are typical within the set of all such observers.
6. A typical observer is likely to be in one of the simulations (at some level), rather than a member of the top-level civilization.
7. Therefore, we probably live in a simulation.
Of course one is welcome to poke holes in any of the steps of this argument. But let’s for the moment imagine that we accept them. And let’s add the observation that the hierarchy of simulations eventually bottoms out, at a set of sims that don’t themselves have the ability to perform effective simulations. Given the above logic, including the idea that civilizations that have the ability to construct simulations usually construct many of them, we inevitably conclude:
We probably live in the lowest-level simulation, the one without an ability to perform effective simulations. That’s where the vast majority of observers are to be found.
Hopefully the conundrum is clear. The argument started with the premise that it wasn’t that hard to imagine simulating a civilization — but the conclusion is that we shouldn’t be able to do that at all. This is a contradiction, therefore one of the premises must be false.
This isn’t such an unusual outcome in these quasi-anthropic “we are typical observers” kinds of arguments. The measure on all such observers often gets concentrated on some particular subset of the distribution, which might not look like we look at all. In multiverse cosmology this shows up as the “youngness paradox.”
Personally I think that premise 1. (it’s easy to perform simulations) is a bit questionable, and premise 5. (we should assume we are typical observers) is more or less completely without justification. If we know that we are members of some very homogeneous ensemble, where every member is basically the same, then by all means typicality is a sensible assumption. But when ensembles are highly heterogeneous, and we actually know something about our specific situation, there’s no reason to assume we are typical. As James Hartle and Mark Srednicki have pointed out, that’s a fake kind of humility — by asserting that “we are typical” in the multiverse, we’re actually claiming that “typical observers are like us.” Who’s to say that is true?
I highly doubt this is an original argument, so probably simulation cognoscenti have debated it back and forth, and likely there are standard responses. But it illustrates the trickiness of reasoning about who we are in a very big cosmos.
The author moved the goal-posts on the initial premise from "it is easy to imagine simulating civilizations" to "it is easy to simulate civilizations" and apparently doesn't expect the audience to notice. He's changed the initial assumption from a reasonable and provable one to one which has been designed to be contradictory.
Jesus, I hope none of these people are actually employed in the hard sciences, they've got a weaker grasp on experimental design, objectivity, and logic than I do, and I took a degree in Liberal fucking Arts. Like what the hell, they're not even trying to pretend that they're acting in good faith.