Why Experts (and We) Fear AI

By Shlomo Maital   

  Recently, Elon Musk and Steve Wosniak (co-founder of Apple) were among co-signers of a letter warning of the dangers of AI, and calling for suspending all work on it for six months.

  Among the co-signers was Dr. Stewart Russell, among the world’s top experts on AI.  He is professor of computer science at Berkeley,  director of the Coley Center for Ethics Science and the Public there, and  author of the book, Human Compatible Artificial Intelligence and The Problem of Control.

    He was interviewed on the latest edition of Science Friday by Ira Flatow.  Here is a summary.  (Caution:  This blog is much longer than usual, 984 words).

     The real danger of AI?   What happens if we develop a super-intelligent AI, that is so much smarter than we are, it can simply negate any of our efforts to control or restrain it.  Science fiction has dealt with this in great detail.  Here is Prof. Russell’s views:

       “In my view, the AI systems that are currently being developed and the ones that have been released recently based on a technology called large language models, represent a type of technology that is intrinsically very difficult to understand and very difficult to guarantee that it will behave in a safe way. So in a very immediate sense, it presents risks, not the sort of apocalyptic risks of taking over the world and extinguishing the human race, but real risks. For example, last week in Belgium, a man was reported to have committed suicide directly as a result of his relationship with one of these chatbots, which was actually advising him and as it were, holding his hand, uh, while he was in the process of committing suicide. The reason why these systems are, are very hard to provide any guarantees for is that they are enormous black boxes.”

    “A large language model is something that very simply predicts the next word given the sequence of proceeding words in a text or in a conversation.   and  you can use that for an interactive conversation. If you put in a question, then it will start generating words that look like an answer. And how do you make them? You start with the blank slate of about a trillion parameters in an, in an enormous, what’s called neural network. You do about a billion trillion random modifications of those parameters to try to get that network to become very good at predicting the next word from a training set that is maybe 20 trillion words, which is roughly comparable to all the books that the human race has ever written in the history of civilization. So that system, when you interact with it, displays remarkable abilities. And I don’t wanna disparage it in the sense that it can provide lots of benefits for users, for companies, but it’s a black box.

     “We do not understand anything about how it works. And the only way we have to get it to behave itself, for example, not to advise people on how to commit suicide, is to essentially say bad dog or good dog. And that’s the process that open AI, the creators of G P T 4  went through to try to get it to behave itself. Um, they just hired a lot of people who would engage in lots of conversations, and every time it did something they don’t like, they would say bad dog. And if it produce a good answer, they would say, good dog. And then hopefully the system would adapt its parameters to produce bad behavior less often. And they proudly announced that in terms of these forbidden things like advising people to commit suicide, telling people how to make chemical weapons, uh, giving unlicensed medical advice, that it was 29% better than the previous iteration of their system.

  “But 29% better is still a very long way from perfect because they have actually no control over it. So we are simply asking that before you get to deploy a system that’s going to affect the lives of millions or even billions of you take sensible precautions to make sure that it doesn’t present undue risks and that it remains within predictable guidelines and so on. So that’s the real reason behind this request for a moratorium, I think there are longer term issues at stake here, not from the present systems, but from future generations of AI systems that may be much more powerful still. And they present correspondingly much greater risks.

   “Theoretically we don’t know when that type of system, which we call sometimes artificial super intelligence, we don’t know when that’s going to arrive. But if it does arrive within our current approach to how we build AI systems, in particular these black boxes, we would have no way of ensuring that it’s safe in the sense that its behavior is actually aligned with what the humans want the future to be like. And then you’re basically setting up a chess match between us and a system that’s actually much more intelligent than us and has already thought of every possible countermeasure we could try,  and so that’s in a real sense, the loss of human control over the future. So that’s the risk that Stephen Hawking is talking about.

” I want to emphasize the current systems do not present that risk as far as we know, to the extent that we understand them at all, which is not very much. We think they have some fundamental limitations on their ability to plan their future activities. But at the rate of progress we’re seeing in AI, we need actually to develop methods to ensure that when we build systems that are more powerful than us, we somehow retain power over them forever. If that sounds like a  difficult problem, it’s because it IS a difficult problem.”