Published on Monday, March 13, 2000 in the San Francisco Chronicle
Scientist Is Fearful of Computer Mutiny:
Sun Micro Co-Founder Says Replicating Robots Could Replace Humans
by Joel Garreau
A respected creator of the Information Age has written an extraordinary critique of accelerating technological change in which he suggests that new technologies could cause "something like extinction'' of humankind within the next two generations.
The alarming prediction, intended to be provocative, is striking because it comes not from a critic of technology but rather from a man who invented much of it: Bill Joy, chief scientist and co-founder of Sun Microsystems Inc., the leading Web technology manufacturer.
Joy was an original co-chairman of a presidential commission on the future of information technology. His warning, he said in a telephone interview, is meant to be reminiscent of Albert Einstein's famous 1939 letter to President Franklin Delano Roosevelt alerting him to the possibility of an atomic bomb.
In a 24-page article in the Wired magazine that will appear on the Web tomorrow, Joy says he finds himself essentially agreeing, to his horror, with a core argument of the Unabomber, Theodore Kaczynski -- that advanced technology poses a threat to the human species.
``I have always believed that making software more reliable, given its many uses, will make the world a safer and better place,'' Joy wrote in the article, which he worked on for six months. ``If I were to come to believe the opposite, then I would be morally obligated to stop this work. I can now imagine that such a day may come.''
Joy enjoys a level-headed reputation in the industry. ``Nobody is more phlegmatic than Bill,'' said Stewart Brand, an Internet pioneer. ``He is the adult in the room.''
Joy is disturbed by a suite of advances. He views as credible the prediction that by 2030, computers will be a million times more powerful than they are today. He respects the possibility that robots may exceed humans in intelligence, while being able to replicate themselves.INEXPENSIVE SMART MACHINES
He points to nanotechnology -- the emerging science that attempts to create any desired object on an atom-by-atom basis -- and agrees that it has the potential to allow inexpensive production of smart machines so small they could fit inside a blood vessel. Genetic technology, meanwhile, is inexorably generating the power to create new forms of life that could reproduce.
What deeply worries him is that these technologies collectively create the ability to unleash self-replicating, mutating, mechanical or biological plagues. These would be ``a replication attack in the physical world'' comparable to the replication attack in the virtual world that recently caused the shutdowns of major commercial Web sites.
``If you can let something loose that can make more copies of itself,'' Joy said in a telephone interview, ``it is very difficult to recall. It is as easy as eradicating all the mosquitoes: They are everywhere and make more of themselves. If attacked, they mutate and become immune. . . . That creates the possibility of empowering individuals for extreme evil. If we don't do anything, the risk is very high of one crazy person doing something very bad.''
What further concerns him is the huge profits from any single advance that may seem beneficial in itself.
``It is always hard to see the bigger impact while you are in the vortex of a change,'' Joy wrote. ``We have long been driven by the overarching desire to know that is the nature of science's quest, not stopping to notice that the progress to newer and more powerful technologies can take on a life of its own.''
Finally, he argues, this threat to humanity is much greater than that of nuclear weapons because those are hard to build. By contrast, he says, these new technologies are not hard to come by. Therefore, he reasons, the problem will not be ``rogue states, but rogue individuals.''
Joy acknowledges that to some people, this may all sound like science fiction. ``After Y2K didn't happen,'' he said, ``some people will feel free to dismiss this, saying everything will work out.''
Joy is less clear on how such a scenario could be prevented. When asked how he personally would stop this progression, he stumbled. ``Sun has always struggled with being an ethical innovator,'' he said. ``We are tool builders. I'm trailing off here.''
©2000 San Francisco Chronicle