- Those who, for one reason or another, are critical of the study of existential risk - as presented in
Chapter 8, Section 10.3, and several other parts of this book - tend to rejoice in comparing that study to Pascal's Wager. A recent blog post
by Swedish bioethicist Christian Munthe may serve as an example. Munthe says that "what drives the
argument [for why we should do something to prevent a particular existential catastrophe] is the (mere) possibility of a massively
significant outcome, and the (mere) possibility of a way to prevent that particular outcome, thus doing masses of good", he goes
on to compare this to the (mere) possibility that Pascal's God exists, and he poses the rhetorical question that is also the title of his blog post: "Why aren't existential risk/ultimate harm argument advocates all attending mass?"
Crucial to Munthe's argument is the clumping together of a wide class of very dissimilar concepts as "the (mere) possibility of a massively significant outcome". Unfortunately, his blog post lacks specific pointers to the literature, but it is clear from the context that what is under attack here are publications like Bostrom and Ćirković (2008) and Bostrom (2013, 2014) - all of them heavily referenced (usually approvingly) in the present book. So for concreteness, let us consider the main existential risk scenario discussed in Bostrom (2014), namely
- (S1) The emergence of a superintelligent AI that has goals and values in poor alignment with our own and that wipes us out as a result.
First, however, consider Munthe's parallel to Pascal's Wager, and note that Pascal didn't have in mind just any old god, but a very specific one, in a scenario that I take the liberty of summarizing as follows:
- (S2) The god Yahweh created man and woman with
original sin. He later impregnated a woman with a child that was in some sense also himself. When the child had grown up, he sacrificed it (and thus, in some sense, himself) to save us from sin. But only those of us who worship him and go to church. The rest will be sent to hell.
- there are innumerable possible versions of the god that lures you with threats and promises of damnation and salvation, and what that particular god may demand in return, often implying a ban on meeting a competing deity's demands, so the wager doesn't seem to tell you to try to start believing in any particular of all these (merely) possible gods.
- (S3) There exists an omnipotent deity, Baal, who likes atheists and lets them into heaven. The only people whom he sends to hell instead are those who make him jeleaous by worshiping some other deity.
- there seems to be an innumerable amount of thus (merely) possible existential risk scenarios, as well as innumerable (merely) possibly workable technologies that might help to prevent or mitigate each of these, and it is unlikely (to say the least) that we have resources to bet substantially on them all, unless we spread them so thin that this action becomes meaningless.
The crucial point is this. There are cases where a phenomenon seems to be possible, although we do not yet understand it well enough to be ready to meaningfully assign it a probability - not even an approximate probability. In such cases, we may speak of a "(mere) possibility". But we must not be misled, along with Munthe, by this expression into thinking that this implies that the phenomenon has extremely low probability.