tag:blogger.com,1999:blog-8131794231697217573.post2741154009181229030..comments2024-03-28T09:06:16.955+01:00Comments on Häggström hävdar: The libraries of Babel, Mendel and TuringOlle Häggströmhttp://www.blogger.com/profile/07965864908005378943noreply@blogger.comBlogger22125tag:blogger.com,1999:blog-8131794231697217573.post-69398231381268974522018-02-18T11:25:32.804+01:002018-02-18T11:25:32.804+01:00Douglas Adams speaks of a superintelligent shade o...Douglas Adams speaks of a superintelligent shade of the color blue, which I suppose is supposed to be nonsensical, but it just occurs to me how this might actually be possible. Digitally, a color is represented as a number, for example a combination of the intensities of red, green, and blue. Our standard today is to use eight bits for each, which makes a 24 bit number, not enough to encode a superintelligence, but to encode a precise shade you need arbitrarily many bits. So, if a superintelligence can be encoded digitally, it can be encoded as a shade of blue.ctailhttps://www.blogger.com/profile/10120443153591800477noreply@blogger.comtag:blogger.com,1999:blog-8131794231697217573.post-21043143637036540162015-04-21T11:11:48.132+02:002015-04-21T11:11:48.132+02:00I’d like to point to an argument that I think has ...I’d like to point to an argument that I think has been lacking, at least in the parts of the AGI-discussion that I have seen.<br /><br />Part of the problem of AGI is that there is a big difference between computation and reasoning. This is obvious today but was perhaps not quite anticipated in the early days of AI-optimism.<br /><br />But there is a kind of “universality” of reasoning too, similar to Turing universality, that we mathematicians take for granted because we have become used to it. The idea, going back to ancient Greece, is that everyone can understand every mathematical result that has ever been established. There are research papers that only experts can read, but there aren’t any results that only smart people can understand.<br /><br />To the outside observer, this is manifested in our apparent obsession with proofs. But the reason we insist on proving things is that we often can, which is rather amazing if we pause and think about it. <br /><br />This universality of reasoning does not seem to have become “established” like the Church-Turing thesis of computation. Some results of logic even seem to point in the opposite direction. By the Gödel incompleteness theorem we cannot have a complete theory of integer arithmetic. This seems to indicate that there isn’t a single “right” axiomatisation of standard mathematics, a view that becomes strengthened by artificial lists of axioms like Zermelo-Fraenkel and the weird status of the axiom of choice. <br /><br />Still, we don’t believe that a superintelligent being could know (using some “super-axioms”) that the Goldbach conjecture is true without being able in principle to explain it in terms of earthly mathematics. <br /><br />In my opinion this universality strongly suggests that there is a rather short program that provides a core from which we can start “surfing” the Library of Turing (by adding layers of stuff like Siri, Watson, deep learning or whatever) and eventually reach a monster. <br /><br />I’m talking only about mathematical reasoning here, and there is probably some disagreement about how general this sort of reasoning is. But this shouldn’t be relevant here since the standard argument for an intelligence explosion is about a computer program doing math-stuff like writing and analysing computer programs, or perhaps designing hardware. <br />Johan Wästlundhttps://www.blogger.com/profile/05815242414868499026noreply@blogger.comtag:blogger.com,1999:blog-8131794231697217573.post-7415462886666071772015-04-18T19:31:13.564+02:002015-04-18T19:31:13.564+02:00One thing to always keep in mind when thinking thr...One thing to always keep in mind when thinking through Superintelligence is the immensely, counterintuitively, hardgraspably large value at stake. It is not *merely* the continued existence of human life on earth. It is the prospect of valuable life in *all* reachable parts of the universe *forever* (i.e. for the duration the universe has features that allow for the existence of beings that instantiate such value) versus bad scenarios such as the entire universe filled with suffering forever or universe with no more sentient life at all anywhere forever.Martinnoreply@blogger.comtag:blogger.com,1999:blog-8131794231697217573.post-66417496957512101692015-04-18T17:56:42.296+02:002015-04-18T17:56:42.296+02:00Torsten Ehrenmark skriver i sina memoarer (antagli...Torsten Ehrenmark skriver i sina memoarer (antagligen inspirerad av Borges) om hur han grips av högmod inför sin första skrivmaskin (en Halda om jag minns rätt) där han potentiellt kan skriva de största litterära verken - det gäller bara att trycka på tangenterna i rätt ordning. Gunnar Englundhttps://people.kth.se/~gunnare/noreply@blogger.comtag:blogger.com,1999:blog-8131794231697217573.post-81551219200587342502015-04-18T16:42:44.675+02:002015-04-18T16:42:44.675+02:00The human brain has about 10^11 neurons and about ...The human brain has <a href="http://en.wikipedia.org/wiki/Neuron#Neurons_in_the_brain" rel="nofollow">about 10^11 neurons and about 10^14 synapses</a>. That is not the kind of discrepancy that warrants talk of combinatorial explosion. Olle Häggströmhttps://www.blogger.com/profile/07965864908005378943noreply@blogger.comtag:blogger.com,1999:blog-8131794231697217573.post-31861428378065297542015-04-18T13:14:46.823+02:002015-04-18T13:14:46.823+02:00In response to, " ... as I don't see whe...In response to, " ... as I don't see where the combinatorial explosion would be coming from. "<br /><br />I think if you compare the neuronal structure of C. Elegans to Homo Sapiens, the number of dendrites plays a key role here.<br /><br />It seems to me that you get a combinatorial explosion from increasing the number of neurons, which increases the need for each neuron to have more dendrites.<br /><br />So, rather than just thinking about how many neurons you need to simulate a human brain, you also need to consider the number of synapses.<br /><br />Eli Birdhttp://ebcode.comnoreply@blogger.comtag:blogger.com,1999:blog-8131794231697217573.post-88417391679463796232015-04-17T14:59:44.516+02:002015-04-17T14:59:44.516+02:00No reason for quadratic time, I just wanted to ent...No reason for quadratic time, I just wanted to entertain the idea that brain simulation doesn’t need to be NP-hard (or PSPACE-hard, like n-body simulation) to be considered a problem that can be computationally infeasible already – disregarding the neuroanatomical part. (That‘s the danger of making these calculations – it makes concrete what doesn’t deserve concreteness.)Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-8131794231697217573.post-90845855504232855832015-04-17T14:45:38.141+02:002015-04-17T14:45:38.141+02:00Why quadratic time? Each neuron only has about 100...Why quadratic time? Each neuron only has about 1000 connections, so reading inputs and updating outputs should only take about 1000 operations per neuron per clock tick, for the most generous assumption of neuron simplicity.ctailhttps://www.blogger.com/profile/10120443153591800477noreply@blogger.comtag:blogger.com,1999:blog-8131794231697217573.post-76787289851301587292015-04-17T14:26:01.486+02:002015-04-17T14:26:01.486+02:00Yes, the passage I quoted from Bostrom was to esta...Yes, the passage I quoted from Bostrom was to establish that he doesn’t much care about issues of scaling. I cherry-picked the most blatant of those passages, it just happens to be in the section on brain simulation.<br /><br />At the danger of continuing along a fruitless tangent: I have no idea how computationally difficult brain simulation is. N-body problems of very simple things in physics is much harder than just NP. But one can approximate it, after a fashion. So maybe neurons (which are more complicated that balls, and which we don’t currently know how to describe) happen to be in some sweet spot where (because of physical distances) most interactions can be ignored, so maybe we can get a reasonable simulation in quadratic time.<br /><br />So I’d be willing to entertain the idea that we can kinda-sorta simulate a brain to some degree of accuracy in quadratic time. There are 10^9 or 10^10 neurons in the human brain, so we’re looking at 10^18 operations. You need a billion modern computers to simulate “one iteration” in a second. How fast is the human brain clocked? No idea if that even makes sense to discretise. 100 Hz?<br /><br />So, with these back-of-the envelope calculations, I can simulate Einstein’s brain using 100 billion computers, assuming neuroanatomy ever gives me a model and a description. I only kinda-sorta simulate it, so it’s unclear if my iEinstein can do anything else than drool, intellectually speaking.<br /><br />Now, how much faster would we need to run Einstein before we have something that is even close to a superintelligence? Einstein on 10 times the speed? Is that better than 10 Einsteins? (Say, Princeton’s faculty in the 1940s.) So I need 100 billion computers to simulate Einstein drooling really fast. Compare this to the perfectly pleasant way of producing new brains by having intercourse.<br /><br />Make no mistake: I like to play these games. But whenever I start, my brain just says “why should <i>that</i> be easy?” As I tried to make explicit: the main conclusion of half a decade of research is that most problems are really, really hard. And quadratic time is quite enough.<br /><br />(Disclaimer: we have no idea how to describe the functions of a neuron, much less in one-“operation”-per-clock cycle speed. The entire above calculation assumes many very charitable interpretations of neuroanatomy probably rendering the entire analysis void. Not even mentioning the required technological breakthrough in “how to slice up brains.”)Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-8131794231697217573.post-20802987813871870472015-04-17T10:58:23.627+02:002015-04-17T10:58:23.627+02:00Human brain emulation does have potential bearing ...Human brain emulation does have potential bearing on the prospects of creating a superintelligence. I wholeheartedly recommend reading the book. Olle Häggströmhttps://www.blogger.com/profile/07965864908005378943noreply@blogger.comtag:blogger.com,1999:blog-8131794231697217573.post-5859267800678618972015-04-17T10:48:22.270+02:002015-04-17T10:48:22.270+02:00Ok, but a) if it's in the book Superintelligen...Ok, but a) if it's in the book <em>Superintelligence</em> I suppose it's supposed to have some bearing on that subject? and b) I took Thore's reaction “scaling is the problem” as a general comment on the “it's just scaling” attitude. But I sort of regret getting myself into defending my interpretation of Thore, since it might not be spot-on what he meant.ctailhttps://www.blogger.com/profile/10120443153591800477noreply@blogger.comtag:blogger.com,1999:blog-8131794231697217573.post-2698115203531198692015-04-17T10:39:44.997+02:002015-04-17T10:39:44.997+02:00Aha ctail, you've missunderstood what the Bost...Aha ctail, you've missunderstood what the Bostrom passage quoted by Husfeldt is about. It's not about building a superintelligence, it's about emulating a human brain. Olle Häggströmhttps://www.blogger.com/profile/07965864908005378943noreply@blogger.comtag:blogger.com,1999:blog-8131794231697217573.post-38230114721044226312015-04-17T09:10:53.380+02:002015-04-17T09:10:53.380+02:00To do the copying you have to have something to co...To do the copying you have to have something to copy, so then you need to already have the superintelligent monster. Nothing indicates that copying a human brain (if it can be done) would bring anything spectacular. It would be an interesting and perhaps useful experiment, however, if it can be done. I think it would tell us a lot about ourselves.ctailhttps://www.blogger.com/profile/10120443153591800477noreply@blogger.comtag:blogger.com,1999:blog-8131794231697217573.post-66086969632266077342015-04-17T06:30:48.421+02:002015-04-17T06:30:48.421+02:00Think that the way the human brain project in EU i...Think that the way the human brain project in EU is a bit telling... https://www.humanbrainproject.eu/brain-simulation-platform1 which I had said so but I did only think it... making smart code for simulations of complex things is harder then it might seam. My personal gut feeling that is based on basically nothing :) sorry for taking space... is that it is really hard to say which way things are going. Could haven an AI this century or we might not... how do you set a good Bayesian test for that? Perhaps it is in your book? :)Magnushttps://www.blogger.com/profile/01617272924116099306noreply@blogger.comtag:blogger.com,1999:blog-8131794231697217573.post-78500568232926409342015-04-17T06:22:36.308+02:002015-04-17T06:22:36.308+02:00Well, it works better but to me it is far from AI ...Well, it works better but to me it is far from AI and that kind of is the "hard question"? Will just doing enough algorithms make it... ofc we do not know... but it is a valid point imho.Magnushttps://www.blogger.com/profile/01617272924116099306noreply@blogger.comtag:blogger.com,1999:blog-8131794231697217573.post-17632717928410074092015-04-16T20:28:49.190+02:002015-04-16T20:28:49.190+02:00Nah... the task of setting up the initial state of...Nah... the task of setting up the initial state of the emulation looks to me a lot more like copying (taking essentially linear time) than like combinatorial optimization. Olle Häggströmhttps://www.blogger.com/profile/07965864908005378943noreply@blogger.comtag:blogger.com,1999:blog-8131794231697217573.post-59732998588040045802015-04-16T18:43:38.363+02:002015-04-16T18:43:38.363+02:00I may have time for a longer response tomorrow, bu...I may have time for a longer response tomorrow, but let me just state that I don’t claim brain <i>simulation</i> takes exponential time. (Neither do I claim that it’s feasible.) Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-8131794231697217573.post-75931591121316142872015-04-16T18:26:16.464+02:002015-04-16T18:26:16.464+02:00The question is: why wouldn't it? For all we k...The question is: why wouldn't it? For all we know, finding just the right configuration (or one of a few right configurations) usually does succumb to a combinatorial explosion. Experience from first few decades of computer science tells us that if it's not immediately obvious how to find something without almost-exhaustive-search, it's usually at least NP-hard.ctailhttps://www.blogger.com/profile/10120443153591800477noreply@blogger.comtag:blogger.com,1999:blog-8131794231697217573.post-58211968923743589892015-04-16T17:13:16.548+02:002015-04-16T17:13:16.548+02:00Om vi kan läsa vikterna på kopplingarna med ett ve...Om vi kan läsa vikterna på kopplingarna med ett verktyg så är problemet antagligen polynomiskt att simulera. Om vi bara kan se att det finns en koppling mellan neuroner, men inte om den är positiv/negativ eller dess storlek så måste man på något sätt välja dessa vikter. Att träna ett neuralt nätverk (d.v.s. att välja vettiga vikter) är ett NP-problem.<br /><br />Förr eller senare lär dock ändå tekniken bli tillräckligt bra för att mäta hur stark en koppling mellan två neuroner är. Simulationen lär knappast vara värre än om varenda neuron är kopplad till alla andra neuroner, vilket är O(n²) i tidskomplexitet.<br /><br />Vill för övrigt tacka dig Olle för att ha skrivit "Slumpens skördar". Jag tror att den boken har inspirerat mig, för mitt kandidatarbete (här på Chalmers) handlar om perkolation! Mycket intressant blogg, jag är besviken att jag inte upptäckt den tidigare.Anonymousnoreply@blogger.comtag:blogger.com,1999:blog-8131794231697217573.post-3712155494178102742015-04-16T16:10:47.493+02:002015-04-16T16:10:47.493+02:00Why would the problem of finding "the layout ...Why would the problem of finding <i>"the layout of the brain emulation graph and its connection to its I/O devices"</i> succumb to a combinatorial explosion?Olle Häggströmhttps://www.blogger.com/profile/07965864908005378943noreply@blogger.comtag:blogger.com,1999:blog-8131794231697217573.post-86363083490579181752015-04-16T15:58:15.113+02:002015-04-16T15:58:15.113+02:00Once you have the ”whole brain” in the computer, I...Once you have the ”whole brain” in the computer, I suppose the time for running it for <em>m</em> moments (where a ”moment” is a time span short enough for the brain to regarded as incapable of changing state in any way noticeable to any part of itself) would be <em>nm</em>, where <em>n</em> is its number of neurophysically atomic parts. This should be enough to sequentialize the parallel operation of the brain.<br /><br />But the first problem is getting to that state. How to find the layout of the brain emulation graph and its connection to its I/O devices (the body). I hope I'm not misinterpreting Thore's argument, but this is what I take as the analog of finding the right book in the library – the problem that plausibly does not scale.ctailhttps://www.blogger.com/profile/10120443153591800477noreply@blogger.comtag:blogger.com,1999:blog-8131794231697217573.post-149940803691083002015-04-16T15:56:05.836+02:002015-04-16T15:56:05.836+02:00This seems a bit unfounded: "The current tren...This seems a bit unfounded: "The current trend in artificial intelligence, away from symbolic reasoning and towards statistical methods like machine learning, that do not aim to build cognitive models, strikes me as an unlikely pathway." The very reason we are making this shift is that it works better.Patrik Lindenforshttps://www.blogger.com/profile/16644154132810106218noreply@blogger.com