Det finns de som skulle välkomna om AI utplånade mänskligheten

Som jag meddelade i förra månaden har jag lanserat en andra blogg, kallad Crunch Time for Humanity, med explicit fokus helt på AI-risk och AI-säkerhet. Det är mest för att påminna dem av er som är intresserade av sådana frågor (vilket jag antar att de flesta av er som läser Häggström hävdar är, då övriga läsare rimligtvis för länge sedan hunnit ledsna på mitt AI-ältande och dra vidare) om denna andra bloggs existens som jag skriver dessa rader.

I fredags färdigställde jag det hittills mest ambitiösa och kanske viktigaste inlägget på Crunch Time for Humanity, rubricerat Those who welcome the end of the human race. Missa inte det! Det behandlar så kallad AI-successionism - den bland AI-experter bekymmersamt vanligt förekommande idén att det vore bra om mänskligheten dog ut och ersattes av avancerad AI. Så här börjar inlägget:

If Homo sapiens goes extinct, is that good riddance? Personally, I would immediately say no, but not everyone is equally on board with the pro-human view. At 38:18 into the remarkable interview that New York Times journalist Ross Douthat did with Silicon Valley billionaire Peter Thiel this summer, there is the following unnerving passage.

RD: I think you would prefer the human race to endure, right?
PT: Uh...
RD: You’re hesitating.
PT: Well, I don’t know. I would... I would...
RD: This is a long hesitation!
PT: There’s so many questions implicit in this.
RD: Should the human race survive?
PT: Uh... yes.
RD: OK.
PT: But...

Thiel then goes on to talk about transcendence and immortality and overcoming nature with the help of God and whatnot. Perhaps we can take solace in the “yes” that he finally manage to produce. Still, even if we agree with him (as I do) that there are many further questions, including definitional issues about what it means to be a human, lurking underneath Douthat’s original question, there is something deeply disconcerting in Thiel’s hesitation and inability to give a quick and clear affirmative answer to that seemingly straightforward yes/no-question. Is this influential thinker and tech entrepreneur fundamentally not on humanity’s side? Is he a successionist?

A successionist is someone who welcomes the end of the human race, provided we have — in Dan Faggella’s words — a worthy successor. And when the intended worthy successor is an AI, we may speak more specifically about AI successionism. This gives rise to further definitional issues, not just about the meaning of “human”, but also that of “worthy”, and while these meanings are far from settled, I will refrain here from trying to fully pinpoint them. Instead, I will accept that the term “successionism” inherits a good deal of fuzziness from “human” and “worthy”, and that the concept therefore is not entirely black and white but leaves considerable grey areas and room for interpretations. But with that said, it seems clear to me from the above quote that Peter Thiel at least leans deeply into successionism.

There are various more clearcut examples of successionists in or around the AI and tech sphere, and let me mention a few. A prominent one is...

Med denna cliffhanger uppmanar jag er att läsa fortsättningen på Crunch Time for Humanity!