I kind of like Johan Norberg. He is a smart guy, and while I do not always agree with his techno-optimism and his (related) faith in the ability of the free market to sort everything out for the best, I think he adds a valuable perspective to public debate.
However, like the rest of us, he is not an expert on everything. Knowing when one's knowledge on a topic is insufficient to provide enlightenment and when it is better to leave the talking to others can be difficult (trust me on this), and Norberg sometimes fails in this respect. As in the recent one minute and 43 seconds episode of his YouTube series Dead Wrong® in which he comments on the futurology of artificial intelligence (AI). Here he is just... dead wrong:
No more than 10 seconds into the video, Norberg incorrectly cites, in a ridiculing voice, Elon Musk as saying that "superintelligent robots [...] will think of us as rivals, and then they will kill us, to take over the planet". But Musk does not make such a claim: all he says is that unless we proceed with suitable caution, there's a risk that something like this may happen.
Norberg's attempt at immediate refutation - "perhaps super machines will just leave the planet the moment they get conscious [and] might as well leave the human race intact as a large-scale experiment in biological evolution" - is therefore just an invalid piece of strawmanning. Even if Norberg's alternative scenario were shown to be possible, that is not sufficient to establish that there's no risk of a robot apocalypse.
It gets worse. Norberg says that
- even if we invented super machines, why would they want to take over the world? It just so happens that intelligence in one species, homo sapiens, is the result of natural selection, which is a competitive process involving rivalry and domination. But a system that is designed to be intelligent wouldn't have any kind of motivation like that.
Norberg cites Steven Pinker here, but Pinker is just as ignorant as Norberg of serious AI futurology. It just so happens that when I encountered Pinker in a panel discussion last year, he made the very same dead wrong argument as Norberg now does in his video - just minutes after I had explained to him the crucial parts of Omohundro-Bostrom theory needed to see just how wrong the argument is. I am sure Norberg can rise above that level of ineducability, and that now that I am pointing out the existence of serious work on AI motivations he will read at least some of references given above. Since he seems to be under the influence of Pinker's latest book Enlightenment Now, I strongly recommend that he also reads Phil Torres' detailed critique of that book's chapter on existential threats - a critique that demonstrates how jam-packed the chapter is with bad scholarship, silly misunderstandings and outright falsehoods.
From my computer science perspective, Norberg's standpoint does not look dead wrong, but rather sensible. However, I would like to back up a few steps and claim that the very notion of “motivation” in the context of computers is meaningless as far as we have no clear definition of what motivation should even mean in their world. And, I'm sorry but, “intelligence” has the same problem. I agree with Alan Turing on this issue. (Of course, agreeing with Turing is always a rather safe bet.) In the debate in Lund the other day, you said something on the lines of Turing having taken the positions that computers could become intelligent. Dead wrong! Turing says no such thing. In fact, in the article he (brilliantly) discusses the issue, “Computing Machinery and Intelligence”, he dismisses the question of whether computers can think in the first paragraph, since it is too vague to be answered, and then replaces it with another one, which can be answered.
SvaraRaderaAnother vague concept here is “taking over the world”. What does this even mean? It can be defined in a number of ways, and I find it difficult to find one that lets me agree that humans are currently in charge. Humanity doesn't even agree with itself on most issues, so what would it mean for humanity to be in charge. For all practical intents and purposes, if machines are ever to “take over” in any sense, it's fairly obvious that this isn't a risk, but a fact that they are either in the process of doing so or that they already have. (It's less obvious that this means humans losing out in any general sense.) Machines take over without any other motivation than that of the humans deploying them, and regardless of whether we refer to their behavior as intelligent.
There are real issues, real dangers, with the accelerating development towards computer use in a wide range of previously human-only tasks and environments. (As well as fantastic possibilities, of course.) We should discuss these issues, our responsibilities for the consequences of technology etc., without getting stuck in semantic issues about how to project vague human concepts like “intelligence” and “motivation” on computers, and without inventing highly hypothetical apocalyptic scenarios resulting from computers in some sense and in some way acquiring these vaguely defined traits. There are enough apocalyptic scenarios to worry about already.
We seem to repeatedly run into semantic issues, ctail. I would like to get past those, and discuss the world instead. Here are a couple of clarifications:
SvaraRaderaMotivation. I use this pretty much synonymously with goal, which is entirely nonmysterious. Any optimization algorithm has a goal. Even a thermostat has a goal.
Intelligence. Following other contemporary AI futurologists, I take this to mean cross-domain ability of efficient optimization. When Turing distanced himself from the term intelligence, he clearly meant something more vague mysterious. When, in 1951, he wrote...
"My contention is that machines can be constructed which will simulate the behaviour of the human mind very closely. [...] Let us now assume, for the sake of argument, that these machines are a genuine possibility, and look at the consequences of constructing them. [...] It seems probable that once the machine thinking method had started, it would not take long to outstrip our feeble powers. There would be no question of the machines dying, and they would be able to converse with each other to sharpen their wits. At some stage therefore we should have to expect the machines to take control."
...he clearly meant, by "outstrip our feeble powers", something close to the modern AI futurology meaning of intelligence. So yes, I insist on Turing having taken the position that computers could become intelligent, in this sense of the word.
This definition of intelligence is clearly wrong; as soon as we have algorithms for efficient optimization in two different domains (say the domains of facial recognition and of translation?) two such algorithms running in parallel would be "intelligent". And they obviously need not be.
RaderaIntelligence is not a yes/no thing. The machine you suggest would not qualify as particularly intelligent, because facial recognition + translation is not a particularly impressive cross-section of domains.
RaderaBut machines have already outstripped our feeble powers in many ways, and they keep outstripping us in more every day. Why worry about some specific point when they become “intelligent” or “conscious”, or “take over”. That point in time can be defined as something in the past or in the future arbitrarily depending on the definition of those concepts. If you define them so that they essentially mean that the machines become human, I believe that you place this point in time infinitely far in the future, but I also believe that this has very little bearing on the virtues and danger of the technology.
RaderaWe've been down this road several times before, ctail, so I won't pursue it, other than by taking a cheap shot at your position that I just spotted:
RaderaIn your 15:52 comment above, you state that "agreeing with Turing is always a rather safe bet", while now (15:44) you seem to disagree with his claim that "we should have to expect the machines to take control". A rather reckless bet!
It depends entirely on what you mean by “take control”, but it's perfectly sensible to say that the machines already have taken control, i.e., placing the takeover point in the past (but in Turing's future). But I don't always agree with Turing from today's point of view, he couldn't predict everything of course.
RaderaI think that misses the point, you could add any number of additional algorithms to the mix (FR+Translation+Go+Chess+Checkers+Driving+Identifying songs played on the radio). Isn't it still obvious that this parallel algorithm is no more intelligent than its parts are?
SvaraRaderaNo, that is not at all obvious.
RaderaIs it suddenly a lot more intelligent if we break the Translation domain up into the English->French, French->English, English->Russian, etc, etc... domains?
SvaraRaderaOf course not. Obviously it is not a matter of counting the number of domains. Stop trying to be stupid.
RaderaSorry Olle, but I think the stupid part actually is your definition above; I don't see any way of reading it (and your answer) other than that intelligence increases with the number of domains involved (or actually with the number of domain-crosses, so with its square). One might attempt to improve on it by also considering some size of the domains, but then it seems no longer to be about domain crossing at all, only about the area (in some unspecified sense) covered by the algorithm.
RaderaHowever, I think your contention that intelligence is "not a yes/no thing" leads you in the wrong direction: to a completely sliding scale. When, presumably, teaching a lot of students basic probability have you not yourself often said "yes, now you've got it!" (and this happens at a clearly marked moment, it's not a sliding scale)?
It is a sliding scale.
RaderaI mean... insect, mouse, dog, chimpanzee, village idiot, Nobel laureate, HAL, ... of course it is a sliding scale. I don't see how one could deny that other than by subscribing to a quasi-religious mystical view of intelligence which is completely useless for the present discussion (or by being located relatively low on that very scale).
RaderaI did not try to claim that intelligence is a binary property, merely that focusing on that it isn't risked leading you wrong in the context ctail's comment. Also, what I said was more appropriate for the notion of understanding, rather than intelligence. However, assuming that intelligence is some kind of measure on the space of optimization problems as above, your "sliding scale" risks obscuring the fact that there (probably) are several barriers along the way from fruit fly to John von Neumann; in as far as the sliding scale suggests that a von neumann is to a fruit fly like (a cluster of) TPU(s) is to an ENIAC my contention is that it has led you wrong.
RaderaAll right Aaron, it now seems to me that our views are much better aligned than I initially thought. I agree with you about the plausibility of major barriers along the way from fruit fly to von Neumann, but note that that is perfectly consistent with measuring intelligence on a continuous scale.
RaderaSo why did our discussion go off the rails?
I believe that it was caused by me, using the words "cross-domain ability of efficient optimization", giving you (unintentionally) the impression that I claimed to have delivered a precise definition of intelligence. You then did what you could to fill in the left-out details (e.g., by making explicit the counting the number of domains that you thought was a crucial such missing detail). That led you to a imagine a definition that you found silly, and you decided to embark on a combination of reductio arguments and Socratian dialogue to help me understand the definition was silly. Since I failed to notice the unintended ambiguity in my original 16:45 comment, I concluded (erroneously) that you were just being obnoxious and intentionally misreading me. Hence my "Stop trying to be stupid", which is not a good way to move the discussion forward, and for which I apologize.
What I meant in my 16:45 comment to ctail was not at all to claim a precise definition of intelligence, but rather to hand-wave at the sort of non-mysterious instrumental ability to modify the world in the direction prescribed by given goals being the relevant concept here, as opposed to, say, some mysterious kind of Searlian sentient understanding.
I'm going through all this mainly for future pedagogical purposes. I do not want to make the same mistake again. Maybe writing "broadly applicable" rather than "cross-domain" in my 16:45 comment would have been enough to avoid most of the annoyance of the subsequent exchange?
A more general lesson (for both of us) might be to try harder to apply the principle of charitable interpretation.
Very well Olle, the socratic method has its dangers in that it is sometimes indistinguishable from plain trolling. Still though, it seems to me that Turing, in your quoted passages, is not talking about cross-domain/broadly applicable optimization ability but rather about computers learning the one task of understanding human language; which they will then proceed to use to "converse with each other". At this point he expects computers to quickly "outstrip our feeble powers", presumably because the tirelessly working computer can simply read through the whole Cambridge University Library and usurp all the human knowledge gathered there.
RaderaI think Thore Husfeldt remarked in Lund that (in your terminology here) the surprise from recent advances in deep learning is that computers can advance on the optimization scale above without advancing at all on the problem of understanding (human) language (which seems to be stuck where it was in Turing's time: nowhere).
The naive techno-optimism mentioned has always scared me. These optimists don´t seem to understand what it really means if artificial intelligence evolves beyond the level of human intelligence. (I would not bet on that "super-machines" will leave our planet. That´s just stupid wishful thinking)
SvaraRaderaWhen media reports on the future of AI, they seem to focus on either 1) the job losses associated with AI or 2) how wounderful our "AI-supported" future world will be. Techno-liberals (like Norberg) seem to ignore 1) and embrace 2). However, they never seem to think about what it really means if we create the Paperclip Maximizer...
Sorgligt att se Norberg upprepa Pinkers misstag! Men inte förvånande.
SvaraRaderaSpaning: det libertarianska ekosystemet kring CATO institute (där Norberg ingår) har länge stöttat klimatförnekande. Nu verkar de nysatsa på AI-riskförnekande. Detta kommer att sippra ner även till svenska klimatförnekarkretsar.
Så kan det vara. Seth Baum har två färska och intressanta uppsatser om fenomenet: Superintelligence Skepticism as a Political Tool och Countering Superintelligence Misinformation.
RaderaTack för länkarna, mycket intressant!
RaderaBara tanken på ett scenario a la Bostroms vulnerable world hypothesis där enda alternativet till undergång är superradikal reglering och övervakning av ekonomi, information, beteende osv torde generera en ryslig emotionell kris för den som identifierar sig starkt med libertariansk ideologi.
Farlig negativ feedback loop också: ju starkare AI desto starkare verktyg för att desinformera om AI och AI-risker.