It may seem immodest of me to say so as an organizer of the event, but I do think that the symposium
Emerging Technologies and the Future of Humanity, held at the
Royal Swedish Academy of Sciences (KVA) in Stockholm last Monday, was a success. The meeting saw illuminating and insightful discussion of possible prospects and consequences of a number of radical emerging and future technologies, thereby achieving the ambition stated beforehand:
A number of emergening and future technologies have the potential to transform - for better or for worse - society and the conditions for humanity. These include, e.g., geoengineering, artificial intelligence, nanotechnology, and ways to enhance human capabilities by genetic, pharmaceutical or electronic means. In order to avoid a situation where in effect we run blindfolded at full speed into unknown and dangerous territories, we need to understand what the possible and probable future scenarios are, with respect to these technological developments and their potential positive and negative impacts on humanity. The purpose of the meeting is to shed light on these issues and discuss how a more systematic treatment might be possible.
Discussions focused not only on possible outcomes, but to some exent also on what we might do to improve the odds of an outcome favorable to humanity. Some of the speakers mentioned the possibility of prohibiting certain lines of research judged to pose particularly severe risks for global catastrophes - not as the only way forward, but as one of several possible strategies worth considering; in particular, Roman Yampolskiy, in his talk based on the paper
Responses to Catastrophic AGI Risk: A Survey coauthored with Kaj Sotala, discussed a very wide range of such strategies for the case of artificial intelligence. Still, the mere mentioning of the possibility of a ban seemed to be enough to agitate a couple of members of the audience, including science journalist Waldemar Ingdahl, who
tweeted about it. In an attempt to elicit a clarification of his point of view, I
responded to his tweet, leading to the following discussion:
WI: Many calls to shut down free research at KVA today
OH: Are you agitated by that, or just making a slightly amusing observation?
WI: like others in the audience and some of the speakers at the KVA event, I found calls to shut down free research worrying
OH: Shutdowns should not be done frivolously, but reflecting on it when the future of humanity is at stake should not be taboo.
OH: Here's my take on it: http://haggstrom.blogspot.se/2011/10/angaende-akademisk-frihet-en.html
WI: shutdowns of free research tend to spread to other areas, see the fate of Swedish GMO and biotech research. That's worrying.
OH: Yes, it's worrying. This worry needs to be weighed against others, e.g., new research leading to extinction of humanity.
Unfortunately, Ingdahl left our exchange at that point. I say "unfortunately", because it is still not clear to me what Ingdahl's position on this issue really is. I see a number of possibilities, including the following, for how he, depending on his position, might respond:
(1) Granted, there may be conflicting ideals to balance there. I just wanted to remind you guys that there are serious downsides to banning research.
(2) Why do you keep pestering me about this? I've already demonstrated a downside to banning research. Such a demonstration is enough to show that banning research is wrong.
(3) Freedom of research is more important than the survival of humanity.
(4) As a liberal (or a libertarian) I consider freedom, including academic freedom, a fundamental thing that I am under no circumstances willing to negotiate away.
(5) The future will always be uncertain and dangerous, and we will never know enough about what a future technology can bring us to warrant a ban.
If Ingdahl's response is (1), then we are in fact in agreement. Response (2), on the other hand, is a simple (but, it seems to me, not uncommon) mistake: the fact that a course of action A has downsides is not automatically a decisive argument against A, because
not doing A might have even worse downsides. Response (3) reflects a value judgement that I strongly disagree with - while I do think freedom of research is an important thing, I think the survival and long-term prospects of humanity is
even more important. Response (4) reflects a position that can be considered a special case of (3), and that involves a praticular mistake, namely the failure to recognize that an unconditional call for the simultaneous enjoyment of all kinds of freedom collapses due to the observation (
made earlier on this blog) that various freedoms are in conflict with each other. In this case, there may, e.g., be a conflict between on one hand freedom of research, and on the other hand the freedom to live on without being slaughtered by a
killer robot or a synthetic killer virus.
Response (5) has some merit, as the consequences of future radical technologies
are notoriously difficult to predict, but I think its conclusion goes way too far. Given
the enormity of what's at stake, I think we have an imperative to try to act with as much foresight as we possibly can, which is precisely why discussions like those at the KVA meeting last Monday (and research projects like those of its main speakers) are so badly needed. And it is certainly
not the case that we can
never know
anything about future technology and its consequences. We do not know at present what the best course of action is, but we should be openminded in trying to figure out an answer. To blindly declare at this stage that we will never find ourselves in a position that calls for a research ban seems just silly to me.
It may of course very well be that I've totally misunderstood Ingdahl, and that his actual position on this issue resembles none of the five alternatives I've sketched. I hope he will tell us.
