More on the Lemoine affair

My blog post two days ago about Google engineer Blake Lemoine who has been put on paid administrative leave for breaking the company's confidentialty rules was written in a bit of a haste, ignoring what I now think may be the two most important aspects of the whole story. I will make up for that omission here, but will not repeat the background, for which I refer back to that earlier blog post. Here are the two aspects:

First, Lemoine is a whistleblower, and whistleblowing tends to be personally very costly. But we very much need whistleblowers, and due to this externality mismatch we also need society to treat its whistleblowers well - even in cases (such as, I suspect, the one at hand) where the message conveyed turns out ultimately wrong. While I do not have any concrete suggestion for what a law supporting this idea should look like, I do believe we ought to have such laws, and in the meantime it is up to each of us to be supportive of individual whistleblowers. Our need for them is greater in Big Tech than in perhaps any other sector, because by responding disproportionally to their commercial incentives rather than to the common good, these companies risk causing great harm:

• this is already happening (see this recent text by Jonathan Haidt), and
• things can get orders of magnitude worse (see this recent text by Eliezer Yudkowsky).

Second, while (as I've said) Lemoine is probably wrong about the AI system LaMDA having achieved consciousness, it is extremely important that we do not brush the issue of AI consciousness permanently aside, lest we otherwise risk creating atrocities, potentially on a scale that dwarfs present-day meat industry. Therefore, the dogmatic attitude of his high-level manager Jen Gennai (Director of Responsible Innovation at Google) that Lemoine describes is totally unacceptable:

When Jen Gennai told me that she was going to tell Google leadership to ignore the experimental evidence [about LaMDA being sentient] I had collected I asked her what evidence could convince her. She was very succinct and clear in her answer. There does not exist any evidence that could change her mind. She does not believe that computer programs can be people and that’s not something she’s ever going to change her mind on.

The possibility of AI consciousness needs to be taken seriously, and it is an issue that can escelate from the hypothetical and philosophical to actual reality sooner than we think. As I remarked in my previous blogpost, AI futurology and AI safety scholars have tended to ignore this issue (largely, I believe, due to its extreme difficulty), but a notable recent exception is the extraordinarily rich paper Propositions Concerning Digital Minds and Society by Nick Bostrom and Carl Shulman. Among its many gems and deep insights, let me quote a passage of particular relevance to the issue at hand:

• Training procedures currently used on AI would be extremely unethical if used on humans, as they often involve:
  • No informed consent;
  • Frequent killing and replacement;
  • Brainwashing, deception, or manipulation;
  • No provisions for release or change of treatment if the desire for such develops;
  • Routine thwarting of basic desires; for example, agents trained or deployed in challenging environments may possibly be analogous to creatures suffering deprivation of basic needs such as food or love;
  • While it is difficult conceptually to distinguish pain and pleasure in current AI systems, negative reward signals are freely used in training, with behavioral consequences that can resemble the use of electric shocks on animals;
  • No oversight by any competent authority responsible for considering the welfare interests of digital research subjects or workers.
• As AI systems become more comparable to human beings in terms of their capabilities, sentience, and other grounds for moral status, there is a strong moral imperative that this status quo must be changed.
• Before AI systems attain a moral status equivalent to that of human beings, they are likely to attain levels of moral status comparable to nonhuman animals—suggesting that changes to the status quo will be required well before general human-level capabilities are achieved.
  • The interests of nonhuman animals are violated on a massive scale in, for example, factory farms, and there is a strong case that this is morally wrong.
  • Nevertheless, there are some systems in place to limit the harm and suffering inflicted on animals (e.g., minimum standards for cage size, veterinary care, outlawing of various forms of animal abuse, the “three Rs” in animal experimentation, etc.).
  • Digital minds that are morally comparable to certain nonhuman animals should ideally have protections similar to those that ought to be extended to those animals (which are greater than those that are at present actually extended to farmed animals).
• Some research effort should be devoted to better understand the possible moral status, sentience, and welfare interests of contemporary AI systems, and into concrete cost-effective ways to better protect these interests in machine learning research and deployment.