söndag 23 april 2023

Are large language models intelligent? Are humans?

Let's say you are in doubt as to whether I am intelligent. Here, by ''intelligent'', I don't mean ''above average intelligent'' or ''as intelligent as one usually expects from a mathematics professor'' but something much more modest. The question you ask is whether I am intelligent at all. Is there anything at all going on inside my head that deserves the label intelligence?

To find out whether I am capable of intelligent reasoning, you decide to test me, and ask the following:
    Michael is at that really famous museum in France looking at its most famous painting. However, the artist who made this painting just makes Michael think of his favorite cartoon character from his childhood. What was the country of origin of the thing that the cartoon character usually holds in his hand?
And here is my reply:
    The most famous painting in the Louvre is the Mona Lisa. The artist who made the Mona Lisa is Leonardo da Vinci. Leonardo da Vinci is also the name of the main character in the cartoon Teenage Mutant Ninja Turtles. Leonardo da Vinci is from Italy. The thing that Leonardo da Vinci usually holds in his hand is a katana. The country of origin of the katana is Japan. The answer is ''Japan''.
So what would be your verdict? Is this an example of intelligent reasoning?

Obviously yes: this is a clear example of reasoning in several steps, and while the reasoning chain is not advanced enough to count as the mark of genius, clearly some intelligence must have been employed to produce the answer. Any other answer would be an insult, not just of me personally but of the very notion of intelligent reasoning.

But let us now change the story a bit, by taking the subjet of your testing to be not me, but instead an AI of the kind known as a large language model. Now you might be tempted to give a different verdict, and insist that the reasoning about the Louvre and Mona Lisa and Leonardo and so on is not actually real reasoning, but rather a fake kind of reasoning based on statistical pattern matching or some such. In fact, the above answer was given by an LLM known as PaLM over a year ago (which feels like a lifetime given the current breakneck speed of AI development).

Many people employ such double standards in judging the presence of intelligence in other people vs in AIs, and reject the idea the LLMs might be even the slightest bit intelligent even in the face of them making such impressively well-reasoned statements as the PaLM/Leonardo reply above, or any of the various much more impressive achievements that we see GPT-4 producing today, a year later. I think it is wrong to be so dismissive, but I admit that in principle such an attitude can be justified as long as one has an argument that (a) shows that AIs of the type at hand simply cannot exhibit real intelligence, at the same time as it (b) doesn't lend itself to deducing the same conclusion about humans. I have yet to see a principled argument against LLM intelligence that achieves (a) while avoiding the generalization to humans indicated in (b). Many attempts have been made at arguments achieving (a), and following is a list of the most common ones, but in each case I will show how the same logic applies to rule out intelligence of humans, and since by any reasonable definition of intelligence humans do have (at least some of) that property we obtain a reductio ad absurdum, so the argument must be rejected.
  • LLMs sometimes say dumb things, so they lack the common sense that is crucial for intelligence.
  • LLMs are just matrix multiplication (along with nonlinear transformations) with coefficients chosen using statistical methods.
  • LLMs only predict the next word.
  • LLMs lack a world model.
  • LLMs have no grounding of their symbols.
  • LLMs lack creativity.
  • LLMs lack consciousness.
Let me go over these seven arguments one at a time, in a tiny bit more detail, and indicate how they generalize to humans.

1. LLMs sometimes say dumb things, so they lack the common sense that is crucial for intelligence.

So anyone who has ever said anything dumb is automatically altogether devoid of intelligence? Well, with such a harsh criterion my intelligence would also be zero, as would (I presume) also the reader's. That's dumb.

It may however be that proponents of this agrument against LLM intelligence (such as Gary Marcus) mean it in a somewhat nore nuanced way. Perhaps they do not literally mean that a single dumb thing someone says does not rule out their intelligence, but rather that the greater prevalence of dumb things in ChatGPT's output than in mine shows that it is not as intelligent as me. Note, however, that such an argument points towards not a qualitative difference but a quantitative one, and therefore cannot be invoked to support the idea that ChatGPT has zero intelligence. Note also that such a comparison depends on the selection of tasks to test: while it is certainly possible to put together a collection of cognitive tasks where I outperform ChatGPT, it is also possible to do it in a way that achieves the opposite results; this greatly complicates the issue of whether it is reasonable to claim that ChatGPT is less intelligent than I am.

Let me finally stress that the term "common sense" is perhaps better avoided, as it is likely to confuse more than it enlightens. It tends to serve as a catch-all term for everything that humans still do better than AI, so that the phrase "AIs lack common sense" will continue to apply right until the moment when AI outperforms us at everything. I've written about this at greater length elsewhere.

2. LLMs are just matrix multiplication (along with nonlinear transformations) with coefficients chosen using statistical methods.

I think the remark about statistical methods here is just a red herring: why in the world would that way of setting the coefficients count against the AI being intelligent? (I only mention it here because it was recently emphasized in an op-ed by three colleagues attacking my view of AI.)

The purpose served by the part about matrix multiplication is to point out that the AI is built up from simple components that are themselves totally dumb. But the same thing goes for me - my brain is built out of atoms, each of which in itself totally lacks intelligence. So it seems to be possible to build intelligent systems out of entirely unintelligent components, whence the "LLMs lack intelligence because matrix multiplication" argument doesn't work.

3. LLMs only predict the next word.

This is the wide-spread "it's just glorified autocomplete" objection to LLM intelligence. However, any claim that LLMs lack intelligence because they do no other work than to predict the next word in a text is based on a fundamental confusion between what the LLM is trained to do and what it then actually does. The analogous confusion applied to humans would be to say that since the human speices was trained by biological evolution, all we ever do is to maximize inclusive fitness (i.e., maximizing the number of fertile offspring, plus nephews and neices etc properly discounted). Training an agent for one goal sometimes leads to the emergence of other, unintended, goals. When GPT-4 behaves as if it is trying to convince me that wearing a seat belt while in a car is a good idea, it could be tempting to say "no, it's not actually trying to do that, it's only trying to predict the next word", but that would be as silly as dismissing the intentions of a human traffic safety advisor by saying "no, he's not trying to convince me about seat belts, he is merely trying to procreate".

Also, listen to what Ilya Sutskever (chief scientist at OpenAI) says about what GPT-4 level next word prediction entails.

4. LLMs lack a world model.

This seems to me as unsubstantiated as claiming that humans lack a world model. In both cases - human as well as LLM - behavior points quite strongly towards the existence of a world model somewhere in the overwhelmingly complex mess that the information processing device - the brain or the deep learning network - constitutes. For the case of GPT-4, see for instance the unicorn example in Section 1 of the Microsoft report on GPT-4 capabilities. In the case of humans we gladly take such behavior as evidence of the existence of a world model, so what reason might we have to interpret the evidence differently for LLMs?

One asymmetry between humans and LLMs is that we can know through introspection about the existence of world models in humans (or at least the one human that is me can know it for precisely the one human that is me). But to point to this asymmetry as an argument for humans and LLMs being different as regards the existence of a world model is to rig the game in a way that seems to me unacceptable, because introspection has by nature the limitation that it can only teach us about ourselves, not about others (and in particular not about LLMs).

So there must be something else about LLMs that is taken as grounds for rejecting the possibility that they might have world models. This is rarely articulated, but perhaps most plausible (or at least common) line of reasoning here is that since LLMs do not have direct access to the real world, there's no way to have a world model. This brings us to the fifth argument.

5. LLMs have no grounding of their symbols.

The argument here is that unlike humans, LLMs cannot really reason about things in the world, because they have never directly accessed these things. For instance, an LLM may seem to speak about chairs using the word "chair", but since they have never seen (or felt) a chair, they have no clue about what the word actually stands for, and so their reasoning about "chairs" doesn't count as real reasoning.

But what about humans? Do we have direct access to things in the world? Immanuel Kant says no (this is his Ding an sich).

As for myself, the fact that I do not have direct access to things does not seem to prevent me from thinking about them. When I think about the Big Bang for instance, it really is the Big Bang that I think about rather than the phrase "the Big Bang" despite never having experienced it directly, and likewise for things like quarks, sovereignity, unicorns and the number 42.

A defender of the "LLMs have no grounding of their symbols" argument might object that there are other things that I actually can experience, such as chairs and trees and even shadows, and that once I have the words "chair" and "tree" and "shadow" properly grounded in real-world objects I can start building a world model that includes composite and more advanced concepts such as the Big Bang, but that without such a solid start the process can never get going. To this I respond (with Kant) that in fact, I do not have direct access to chairs or trees, because my contact with them is always mediated by light waves or sound waves or simply the signals sent from my various sensory organs to my brain. This is analogous to how an LLM's experience of the world is mediated via text. Looking at this more abstractly, the mediation in both cases is via an information package. Of course there are differences between the two cases, but I fail to see how any of these differences would be of such fundamental importance that it warrants the judgement that there is symbol grounding in one case but not in the other.

6. LLMs lack creativity.

This is an argument that was put forth by one of my two foremost idols in the (pre-)history of AI, namely computer scientist Ada Lovelace, who worked in the mid-19th century - long before the term computer science was first conceived. Together with Charles Babbage, Lovelace worked on some of the first prototypes for computing machinery, and had great visions and admirable foresight regarding what such machines might eventually be able to do. This included such seemingly creative tasks as composing music, but she categorically denied that this or anything else produced by such a machine was true creativity, because anything the machine does has already been laid down into the machine (at least implicitly) by the programmer, so all creative credit should go to the programmer.

Enter my other great idol, Alan Turing. In his seminal and celebrated 1950 paper Computing machinery and intelligence, he objected to Ada Lovelace's argument by pointing out that if we take it seriously, then we can apply essentially the same argument to rule out human creativity. Everything I write in this essay has been caused by a combination of factors external to my mind: my genes, my childhood and education, and all other environmental factor influencing me throughout my life, so if I should happen to say anything original or creative in these lines, credit for that is due not to me but to all these external influences.1

The conclusion that human creativity is impossible is of course crazy, so Turing took this as an indication that the definition of creativity implicit in Lovelace's argument was wrong. Instead, he proposed the definition that someone is creative if they produce something that noone else had anticipated, and he pointed out that with this definition, examples of machine creativity existed already at his time of writing. In 2023, we see such examples every day, from LLMs as well as other AIs.

7. LLMs lack consciousness.

Against the argument that LLMs lack conciousness and therefore are not intelligent I have two separate rebuttals. The first is that the argument conflates two very different kinds of properties. An individual's intelligence is a matter of what he/she/it is able to do, whereas consciousness is about what it feels like to be that individual - or, rather, whether or not being that individual feels like anything at all. A priori, these properties are logically independent, and to postulate otherwise is to create confusion.2

But even if we were to accept that intelligence implies consciousness, the argument that LLMs lack intelligence because they lack consciousness fails, because we simply do not know whether or not they are conscious. If we read up on the history of the philosophy of mind, we will of course find many examples of philosophers arguing that various classes of entities (including the class of digital computers) lack consciousness, but none of these arguments are anywhere near conclusive or even moderately convincing.

Refusing to declare someone intelligent because we do not know they are conscious also puts us in the uncomfortable position of having to declare all humans other than ourselves not intelligent. There is precisely one human whose consciousness I am sure of, namely myself. In all other cases, I am politely assuming consciouness - an assumption that seems not only polite but also highly plausible, but I can't say I know. We simply do not understand the phenomenon of consciouness anywhere near well enough to be able to say how general it is. Your brain is sufficiently similar to mine that it makes sense for me to assume that you, just like me, are conscious. Yet, there are differences between our brains (as evidenced, e.g., by our diverging personalities), and we simply do not know that the consciousness phenomenon is so broadly present that these differences do not matter to the presence of that phenomenon. And likewise, we do not know it is so narrow that it does not extend to some large class of non-human objects including LLMs.3


1) This line of reasoning points, of course, towards the notorious issues of free will, a direction which however I do not want to pursue here.

2) An instructive example of such confusion is John Searle's 2014 erroneous argument for why a robot apocalypse is impossible.

3) One way to try to escape this conundrum would be to say that only creatures with the tendency to declare themselves conscious are conscious, thereby making consciousness (at least for the time being) an exclusively human phenomenon. Except it doesn't, in view of, e.g., the Lemoine affair, where an LLM declared itself conscious. Also, it would rule out dog or horse consciousness - something that I believe few dog or horse owners would accept.

söndag 16 april 2023

Ett bottennapp i AI-debatten

Min av AI-utvecklingen och AI-debatten triggade förhöjda medianärvaro som jag för ett par veckor sedan rapporterade om i den förra bloggposten har fortsatt. Jag tänker inte räkna upp alla framträdanden, men vill peka på de båda var för sig drygt timslånga poddsamtalen på Future Strategist och Rak höger med Ivar Arpi som de tydliga höjdpunkterna. Lyssna gärna på något av dem!

Istället för att orda mer om dessa tänker jag här istället säga lite om det som varit det tydligaste bottennappet, nämligen mitt meningsutbyte i Aftonbladet med egenföretagaren och tidigare riksdagsledamoten Mathias Sundin: Sundins inledande artikel är en attack på det öppna brev från Future of Life Institute som jag är medundretecknare till och har rapporterat om tidigare, och som förordar en paus i träningen av de största AI-modellerna så att det eftersläntrande säkerhetsarbetet skall ha en chans att komma ikapp. Hans huvudinvändning mot brevet är att en sådan paus skulle bädda för Kina att köra om oss i väst i AI-utvecklingen, och han passar också på att banna oss ''AI-alarmister'' för att sakna grund för våra varningar och för att istället föra ''extremt långtgående och krystade teoretiska resonemang som de sedan dramatiskt målar upp som mänsklighetens undergång''.

I mitt svar beklagar jag att Sundin driver
    exakt det slags retorik som riskerar att elda på den farliga kapplöpningen, och fördomsfullt tar för givet att kinesiska bolag skulle vara mindre villiga än amerikanska att ta ansvar och samarbeta kring undvikandet av en AI-apokalyps,
men i övrigt lägger jag merparten av spaltutrymmet på att kort peka på delar av den (alls icke krystade) teoretiska grunden för att betrakta den skenande AI-utvecklingen som ett existentiellt hot mot mänsklighetens överlevnad. Man hade kunnat hoppas att Sundin i sin slutreplik skulle befatta sig med något av detta, men av detta blir det exakt noll och intet, och han nöjer sig istället med att (felaktigt) proklamera att de risker jag framhåller är ''ren spekulation'' och ''en fantasi som inte baseras på några fakta''. Och lika lite som han bygger under dessa påståenden,1 lika lite ger han något skäl för sin bisarra ståndpunkt att ''ChatGPT 4 kan inte tänka och är inte intelligent, inte ens lite''.2

Ingenting i Sundins båda debattinlägg ger någon som helst antydan om att han bemödat sig om att ta del av den forskning han beskriver som ''ren spekulation''.3 Att gå ut i allmän debatt och grundlöst men med myndig stämma och spelad auktoritet döma ut hela forskningsområden utan att kunna något om dem är ett beteende som vi tidigare sett hos exempelvis klimatförnekare och kreationister, och som försämrar signal-brus-förhållandet i debatten på ett direkt skadligt vis.

Jag vill helst tro gott om mina medmänniskor, och antar därför att Sundin gärna skulle höja nivån på sin AI-retorik jämfört med den nuvarande, som är präglad av det som verkar vara hans modus operandi: att ur röven dra fram sakpåståenden och torgföra dessa om de låter som stöd för hans allmänna agenda, helt utan hänsyn till deras eventuella sanningshalt. Men för en sådan nivåhöjning krävs att han lär sig något om forskningsläget inom AI-risk, och för den sakens skull har jag några lästips åt honom:

Han skulle kunna börja med att läsa Stuart Russells bok Human Compatible: Artificial Intelligence and the Problem of Control från 2019, eller min egen Tänkande maskiner: Den artificiella intelligensens genombrott från 2021.4 Dessa böcker ger en allmän bakgrundsteckning, även om de trots att de bara har fyra respektive två år på nacken redan har börjat bli lite föråldrade i vissa delar till följd av den rasande takten i senare års AI-utveckling. Därför kan det vara bra att komplettera med några nyare och lite kortare texter. Här finns mycket att välja på, men två jag verkligen vill rekommendera är AI alignment from a deep learning perspective av Richard Ngo, Lawrence Chan och Sören Mindermann, och Without specific countermeasures, the easiest path to transformative AI likely leads to AI takeover av Ajeya Cotra. Det kan också vara värdefullt för Sundin att ta del av OpenAI:s dokument Planning for AGI and beyond från februari i år, där den framtid de aspirerar på att vara med och skapa diskuteras med ord som dessa:
    The first AGI will be just a point along the continuum of intelligence. We think it’s likely that progress will continue from there [...]. If this is true, the world could become extremely different from how it is today, and the risks could be extraordinary. A misaligned superintelligent AGI could cause grievous harm to the world. [...]

    Successfully transitioning to a world with superintelligence is perhaps the most important—and hopeful, and scary—project in human history. Success is far from guaranteed, and the stakes (boundless downside and boundless upside) will hopefully unite all of us.

Här kan man glädjas åt att de inte hänger sig åt samma nedlåtande syn som Sundin på risker och risktänkande. Icke desto mindre agerar de vanvettigt våghalsigt med allas vår framtid i potten, vilket de med chockerande uppriktighet medger i den tekniska dokumentationen i samband med förra månadens släpp av GPT-4:
    Finally, we facilitated a preliminary model evaluation by the Alignment Research Center (ARC) of GPT-4’s ability to carry out actions to autonomously replicate and gather resources—a risk that, while speculative, may become possible with sufficiently advanced AI systems—with the conclusion that the current model is probably not yet capable of autonomously doing so.

    Further research is needed to fully characterize these risks.

    [Mina kursiveringar.]

Som sagt: AI-paus, anyone?


1) Det närmaste jag hittar sådan underbyggnad från Sundins sida är hans förklaring i en efterföljande Twitterdiskussion att hans uppskattning om ''0% risk att AGI förgör mänskligheten'' är ''baserad på att mänsklighet kommer undvika att något sådant sker''. Vi kommer alltså, om Sundin här har rätt, att lyckas ta oss samman och förhindra en slutlig AI-katastrof. Detta är i själva verket det utfall jag för egen del hoppas på och eftersträvar, men om det skall kunna bli verklighet krävs att vi tar riskerna på allvar. Att på Sundins vis förneka riskernas existens är inte något bra sätt att uppmuntra till det nödvändiga säkerhetsarbetet.

2) I debatten florerar en uppsättning handviftningsargument för varför de förmågor GPT och andra stora språkmodeller uppvisar inte är äkta intelligens, men de har alla det gemensamt att de (med minimala modifieringar) visar att inte heller människor har äkta intelligens. Av detta kan vi dra slutsatsen att argumenten antingen är felaktiga eller att de underförstått laborerar med ett irrelevant intelligensbegrepp. Jag hoppas inom kort få tid till en bloggpost med en kort genomgång av de vanligaste argumenten av denna typ.

3) Inte heller i Sundins bakgrund finns något som tyder på kunskaper i ämnet - Wikipedia anger hans främsta akademiska meritering som ''tidigare lärarstudent''.

4) Om Sundin är intresserad kan jag skicka honom ett signerat exemplar av Tänkande maskiner.

söndag 2 april 2023

Tidningsrubriker i veckan

Aldrig tidigare har jag varit så uppvaktad av journalister som denna vecka. Det handlar givetvis om den skenande AI-utvecklingen. Inledningsvis kom intervjuförfrågningarna som reaktion på min debattartikel i Ny Teknik förra fredagen, men därefter i än högre grad triggat av det öppna brev om en sex månaders AI-paus, initierat av Future of Life Institute, som jag var med och undertecknade. Här, i approximativ kronologisk ordning, en ofullständig lista över rubriker som dessa intervjuer givit upphov till: Vi får se om nästa vecka blir lugnare (för min del alltså - AI-utvecklingen lär inte avstanna).