Myth: “Artificial superintelligence will be too smart to make mistakes.”
Reality: AI researcher and founder of Surfing Samurai Robots, Richard Loosemore thinks that most AI doomsday scenarios are incoherent, arguing that these scenarios always involve an assumption that the AI is supposed to say “I know that destroying humanity is the result of a glitch in my design, but I am compelled to do it anyway.” Loosemore points out that if the AI behaves like this when it thinks about destroying us, it would have been committing such logical contradictions throughout its life, thus corrupting its knowledge base and rendering itself too stupid to be harmful. He also asserts that people who say that “AIs can only do what they are programmed to do” are guilty of the same fallacy that plagued the early history of computers, when people used those words to argue that computers could never show any kind of flexibility.
Peter McIntyre and Stuart Armstrong, both of whom work out of Oxford University’s Future of Humanity Institute, disagree, arguing that AIs are largely bound by their programming. They don’t believe that AIs won’t be capable of making mistakes, or conversely that they’ll be too dumb to know what we’re expecting from them.
“By definition, an artificial superintelligence (ASI) is an agent with an intellect that’s much smarter than the best human brains in practically every relevant field,” McIntyre told Gizmodo. “It will know exactly what we meant for it to do.” McIntyre and Armstrong believe an AI will only do what it’s programmed to, but if it becomes smart enough, it should figure out how this differs from the spirit of the law, or what humans intended.
McIntyre compared the future plight of humans to that of a mouse. A mouse has a drive to eat and seek shelter, but this goal often conflicts with humans who want a rodent-free abode. “Just as we are smart enough to have some understanding of the goals of mice, a superintelligent system could know what we want, and still be indifferent to that,” he said.
.
