What Max Tegmark really said about AI risk

In a recent article in the Australian online magazine Quillette, Swedish AI pundit Mathias Sundin portrays his more framous compatriot Max Tegmark as a doomsayer. He quotes Tegmark as saying the following in the Swedish public radio show Sommar on August 1, 2023:

I’ve been thinking a lot about life and death lately. Now, it’s probably my turn next in my family. But I guess the rest of humanity will perish about the same time — after over a hundred thousand years on our planet. I believe the artificial intelligence that we’re trying to build will probably annihilate all of humanity pretty soon.

The quote is authentic and the translation from Swedish is fine. But look at Sundin's comment immediately after the quote:

There were no ifs or buts and no “10 percent risk” or other disclaimers — just the promise of certain doom.

The careful reader might here notice the contrast between Sundin's talk of “certain doom”, and Tegmark's use of the word “probably” — which normally signifies not certainty but uncertainty.¹ If the reader trusts that Sundin writes in good faith, he will likely conclude that the word “probably” is used here by Tegmark not in its real meaning, but as meaningless sentence filler and random noise, and that if one looks carefully at what else he has to say in the Sommar show, Sundin's summary about “the promise of certain doom” turns out to be accurate.

So let's have a look then, at what Tegmark has to say in Sommar, to determine whether Sundin's summary statement is warranted. (Spoiler: it is not.)

Less than half a minute after the passage quoted by Sundin, Tegmark says the following:

What can we do today, in order to change direction towards a more hopeful and inspiring future?²

This is already an indication that, far from preaching “certain doom”, Tegmark thinks that “a more hopeful and inspiring future” is possible, and wants to engage his listeners in the great project of navigating towards such a future.

Soon after that, Tegmark goes on to compare an AI catastrophe wiping out humanity to a more down-to-Earth and familiar event: the death at old age of his own parents. He feels his parents had full lives, and thinks the possibility of an AI apocalypse compares unfavorably to this:

But if the entire human race dies out because we messed up with AI, I feel in contrast that we have neither lived fully nor died honorably. Rather, it would feel tragic and unnecessary, as if a child makes the mistake of riding its bike across the edge of a precipice, despite repeated warnings. We are so young from a cosmic perspectives, with billions of promising years ahead of us. If we avoid that precipice, our future can become so much better than our past. We are on the verge of finally taking control of our own destiny, and solving many of the problems we humans have so far failed at.³

“Our future can become so much better than our past” are hardly the words of someone who predicts “certain doom”, and the entire passage is on the contrary rather hopeful. Later in the show, Tegmark spells out his hopeful vision about a flourishing future in slighly more detail:

I have talked a lot about the threat from AI, but what happens if we manage to change course so as to end up in a future with a superintelligence which is under our control, and which is neither used maliciously, nor is misaligned or in competition with us? First of all, this will be like Christmas Day for anyone who likes research and technology, as the research done by the superintelligence will be limited not by what humans are able to think up, but only by the laws of nature. Discoveries that I believed as a teenager would take millennia or an eternity might be solved within our lifetimes. My uncle Erik and his daughter Anna both died in cancer, but I am convinced that all cancer can be cured — the problem is just that we humans have so far failed to figure out how. I believe superintelligence would quickly find cures for all diseases, and lift everyone out of poverty, stabilize the climate and solve all the classical global problems we have so far failed at.⁴

A more striking counterexample to Sundin's claim that Tegmark offers “no ifs and buts” in his talk about AI risk would be hard to imagine. And there's more:

What drives my AI activism is not just that I want to avoid problems, but also that I want to preserve these inspiring possibilities.⁵

And this:

We do not need to compete with AI. It is just Moloch trying to fool us into that attitude. We are the ones building AI, and if we make sure to keep control, then it will be us rather than AI who call the shots.⁶

It is clear from these quotes that far from predicting “certain doom”, Tegmark thinks a bright future awaits us if only we get our act together and play our cards right. It is furthermore clear that he doesn't even think this right way of playing our cards necessarily invloves giving up on the project of building superintelligent AI. To the contrary, such AI plays a key role in the grand visions for the future that he paints. It's just that he understands that there are risks involved and that we therefore need to proceed with a suitable level of caution, something that he judges the leading AI developers to lack. Hence his talk about the need to change course.

This is a highly reasonable position (and one that I share),⁷ but conveying it accurately goes counter to Sundin's ambition of painting Tegmark in the worst possible light. Instead, Sundin shamelessly decides to gamble on the readers of Quillette not knowing Swedish and therefore not being able to check his story, and to simply lie about what Tegmark says in Sommar.

What else can be said about Sundin's Quillette article? Well, it is quite bad. As is sadly typical for large parts of the AI debate as a whole and for his own writings in particular, Sundin is not interested in engaging seriously with the arguments of his opponents, and the article is full of inaccuracies and misrepresentations. I spent so much ink in 2023 — one newspaper op-ed and two blog posts — on Sundin's poor writing that frankly speaking I am sick of it, and so I will not say anything more about his Quillette article. I will not even comment on his choice (which is especially bizarre given the long and tedious email exchange we had about it in December last year) to continue to grossly misrepresent the point of my Lex Luthor thought experiment.

This is mostly not about Sundin himself, but more intended as an example of the abysmally low level of discourse among the category of accelerationists and AI risk deniers that he represents. I do worry, however, that him being chosen in December last year as a member of the Swedish government's AI commission is a small sign of governmental dysfunction.

Footnotes

1) The word used in the Swedish original is “antagligen” (01:37 into the radio show), which has very similar connotations in the present context as the English “probably”.

2) The Swedish original, beginning at 02:05: Vad kan vi göra idag, för att ändra riktning mot en mer hopfull och inspirerande framtid?

3) The Swedish original, at 03:48: Men om hela mänskligheten dör ut, för att vi klantat till det med artificiell intelligens, så känner jag däremot att vi varken levt färdigt eller dött värdigt. Snarare skulle det kännas tragiskt och onödigt, som om ett barn av misstag cyklar över kanten på ett stup, trots många varningar. Vi är så unga ur ett kosmiskt perspektiv, med miljarder lovande år framför oss. Om vi undviker det där stupet så kan vår framtid dessutom bli så mycket bättre än vårt förflutna. Vi är på vippen att äntligen ta kontroll över vårt eget öde, och lösa många av de största problem som vi människor hittills gått bet på.

4) The Swedish original, at 54:45: Jag har pratat mycket om hotet från artificiell intelligens, men vad händer om vi lyckas ändra kurs, och hamnar i en framtid med en superintelligens under vår kontroll, som inte används illvilligt, är felriktad eller konkurrerar med oss? Först och främst blir det som julafton för alla som gillar forskning och teknik, eftersom superintelligensens forskning blir begränsad inte av vad vi människor kan lista ut, utan bara av vad som är fysiskt möjligt. Upptäckter som jag som tonåring trodde skulle ta tusentals år eller en evighet skulle kunna ske under vår livstid. Både min morbror Erik och hans dotter Anna dog t.ex. av cancer. Jag är övertygad om att all cancer går att bota, problemet är att vi människor hittills inte lyckats lista ut hur. Jag tror att superintelligens snabbt skulle hitta botemedel för alla sjukdomar, och lyfta alla ur fattigdom, stabilisera klimatet, och lösa alla de klassiska världsproblemen om vi hittills gått bet på.

5) The Swedish original, at 56:10: Min AI-aktivism drivs inte bara av att jag vill undvika problem utan också av att jag vill bevara de här inspirerande möjligheterna.

6) The Swedish original, at 01:04:00: Vi behöver inte konkurrera med AI. Det är bara Moloch som försöker lura i oss det. Det är nämligen vi som bygger AI, och om vi ser till att vi behåller kontrollen så blir det vi och inte AI som bestämmer var skåpet skall stå.

7) For a summary statement of where I stand on AI issues, readers who know Swedish are encouraged to consult the bonus chapter (downloadable for free) in the 2023 edition of my book Tänkande maskiner. For those who prefer English, there are various video recordings of talks I've given that can serve a similar purpose; see here, here and here.