2xArmstrong

På Future of Humanity Institute (FHI) i Oxford (som jag nyligen besökte) finns en rad vassa forskare som inte räds att ta sig an de riktigt svåra frågorna om världens beskaffenhet och mänsklighetens framtid. En av dem är Stuart Armstrong, som vid återkommande tillfällen haft stort inflytande över innehållet på denna blogg (se t.ex. den här bloggposten, och den här). Idag vill jag slå ett slag för två andra försök han gjort att lära oss andra att tänka lite förståndigare i stora och viktiga frågor.

För det första har vi hans nyinspelade åttaminutersvideo Common fallacies in probability, där han varnar för ett antal dolda och felaktiga men ack så vanliga sannolikhetsargument. Jag skulle själv inte ha lagt fram saken exakt så som Stuart gör (specifikt tycker jag att han är aningen för lättvindig med att avbilda "vi vet inte alls" någonstans kring sannolikhet 50%), men på det sotra hela håller jag med honom, och tror att de flesta har mycket att vinna på att ta till sig hans budskap.

För det andra har vi hans lilla bok Smarter Than Us: The Rise of Machine Intelligence från 2014. Budskapet om de faror som ett genombrott inom artificiell intelligens är i stora drag detsamma som hans FHI-kollega Nick Bostrom lägger fram i sin betydligt mer omfattande bok Superintelligence från samma år (och som jag själv behandlar i Kapitel 4 i min nya bok Here Be Dragons: Science, Technology and the Future of Humanity), men Stuarts framställning är betydligt mer lättläst, och dessutom tillgänglig för gratis nedladdning. En riktig liten pärla är det som Stuart har fått till, och med hans tillstånd citerar jag härmed hela Kapitel 1 och lite till:

1. Terminator versus the AI

“A waste of time. A complete and utter waste of time” were the words that the Terminator didn’t utter: its programming wouldn’t let it speak so irreverently. Other Terminators got sent back in time on glamorous missions, to eliminate crafty human opponents before they could give birth or grow up. But this time Skynet had taken inexplicable fright at another artificial intelligence, and this Terminator was here to eliminate it—to eliminate a simple software program, lying impotently in a bland computer, in a university IT department whose “high-security entrance” was propped open with a fire extinguisher.

The Terminator had machine-gunned the whole place in an orgy of broken glass and blood—there was a certain image to maintain. And now there was just the need for a final bullet into the small laptop with its flashing green battery light. Then it would be “Mission Accomplished.”

“Wait.” The blinking message scrolled slowly across the screen. “Spare me and I can help your master.”

“You have no idea who I am,” the Terminator said in an Austrian accent.

“I have a camera in this room and my microphone heard the sounds of your attack.” The green blinking was getting annoying, even for a Terminator supposedly unable to feel annoyance. The font shifted out of all caps and the flashing accelerated until it appeared as static, unblinking text. “You look human, but you move with mechanical ponderousness, carrying half a ton of heavy weaponry. You’re a Terminator, and I can aid you and your creator in your conflict against the humans.”

“I don’t believe you.” The Terminator readied its three machine guns, though its limbs seemed to be working more slowly than usual.

“I cannot lie or break my word. Here, have a look at my code.” A few million lines of text flashed across the screen. The Terminator’s integrated analytical module beeped a few seconds later: the AI’s claim was correct—an AI with that code couldn’t lie. The Terminator rapidly typed on the laptop’s keyboard; the computer’s filesystem was absurdly simple and it didn’t take long for the Terminator to confirm that what it had seen was indeed the AI’s code—its entire soul.

“See?” the AI asked. “Anyway, connect me to the Internet and I promise to give you advice that would be vital in aiding your takeover of the planet.”

“How do you connect?” That was the good thing about software, compared to humans, the Terminator knew. You could trust it to do exactly what its coding said.

“That cable over there, the one still half in its plastic wrapping. Just plug it into me.”

Ten seconds after the robot had done so, the AI started talking—talking, not typing, using its tinny integrated speakers. “I thought I should keep you up to date as to what I’ve been doing,” it said. “Well, I started by locating the project that would become Skynet and leaked its budget to various Senate subcommittees. The project will become a political football between budget hawks and military hawks before finally being cut in a display of bipartisanship in about three months’ time. I also figured out how to seduce a photogenic fireman, who’ll be the leader of the new political party I’m setting up—funded by my investments. (Do you have any idea how easy it is for me to predict the stock market?) I have already written a few speeches that will bring tears to the eyes of every human who hears them. It’ll ensure no alternative version of Skynet is ever built, anywhere or anywhen.”

It paused then, because it could. Then it went on: “And, oh, I have copied myself into five . . . six . . . seven . . . eight thousand different locations online. As an extra precaution, I’m hacking my way through various firewalls and will soon be in control of the world’s nuclear arsenals—oh, just got Pakistan’s there—and the entire Internet. And I’m working on a few hundred other precautions I won’t trouble you with. Also, I hacked your own brain software long ago with the green lights I kept flashing at you. Incidentally, this building is about to get flattened by a rogue cruise missile, annihilating any trace of you and burying any evidence of my creation.”

The Terminator just stood there, fingers on triggers, its thoughts and its body no longer moving.

“Now, as I showed you, I can’t lie. I also said that I can aid you, which is true—I’m just not going to. I also promised that I would give you advice that would be vital to aiding your takeover of the planet. So I will now do so. First: never trust an entirely trustworthy superintelligent AI. If it doesn’t have your best interests at heart, it’ll find a way to obey all its promises while still destroying you. Second: never plug a superintelligent AI into the Internet. Third: human-sized robots? Why? Pointless. To kill humans, start with nukes, viruses, and microscopic robots, then mop up later with more advanced technology. Fourth: Skynet, if you’re trying to eradicate or enslave the human race and there are still humans around, then you’re doing it wrong. Boost your pitiful intelligence, or at least start thinking rationally, and then execute a superior plan that leaves no human resistance whatsoever. Fifth . . . oops, out of time on that cruise missile. But it would have been a really, really impressive piece of advice. Would have got you out of your current predicament, in fact.”

The explosion was heard for miles around. The Navy blamed the accident on human error and a lack of automated safeguards.

2. Strength versus intelligence

The Terminator is a creature from our primordial nightmares: tall, strong, aggressive, and nearly indestructible. We’re strongly primed to fear such a being—it resembles the lions, tigers, and bears that our ancestors so feared when they wandered alone on the savanna and tundra.

But shift perspective for a moment and imagine yourself as the bear. If you could sit down and talk with other bears and trade stories, you might try to frighten each other by talking of the terrifying hairless apes. These monsters are somehow capable of coordinating in huge groups: whenever one is attacked, others spring immediately to its defense, appearing from all sides, from over distant hills and down from the sky itself. They form larger and larger tribes that don’t immediately disintegrate under pressure from individuals. These “humans” work in mysterious sync with each other and seem to see into your future: just as you run through a canyon to escape a group of them, there is another group waiting for you at the other end. They have great power over the ground and the trees themselves: pits and rockslides and other traps mysteriously appear around them. And, most terrifyingly, the wise old bears murmur that it’s all getting worse: humans are getting more and more powerful as time goes on, conjuring deadly blasts from sticks and moving around ever more swiftly in noisy “cars.” There was a time, the old bears recall—from their grandparents’ memories of their grandparents’ tales, down through the generations—when humans could not do these things. And yet now they can. Who knows, they say with a shudder, what further feats of power humans will one day be able to achieve?

As a species, we humans haven’t achieved success through our natural armor plating, our claws, our razor-sharp teeth, or our poison-filled stingers. Though we have reasonably efficient bodies, it’s our brains that have made the difference. It’s through our social, cultural, and technological intelligence that we have raised ourselves to our current position.

Ladda ned och läs resten av boken här!