Scientist Is Fearful of Computer Mutiny
Sun Micro co-founder says replicating
robots could replace humans
Joel Garreau, Washington Post
Monday, March 13, 2000
WASHINGTON -- A respected creator of the
Information Age has written an extraordinary critique
of accelerating technological change in which he
suggests that new technologies could cause
``something like extinction' of humankind within the
next two generations.
The alarming prediction, intended to be provocative, is
striking because it comes not from a critic of
technology but rather from a man who invented much
of it: Bill Joy, chief scientist and co-founder of Sun
Microsystems Inc., the leading Web technology
manufacturer.
Joy was an original co-chairman of a presidential
commission on the future of information technology.
His warning, he said in a telephone interview, is
meant to be reminiscent of Albert Einstein's famous
1939 letter to President Franklin Delano Roosevelt
alerting him to the possibility of an atomic bomb.
In a 24-page article in the Wired magazine that will
appear on the Web tomorrow, Joy says he finds
himself essentially agreeing, to his horror, with a core
argument of the Unabomber, Theodore Kaczynski --
that advanced technology poses a threat to the human
species.
``I have always believed that making software more
reliable, given its many uses, will make the world a
safer and better place,' Joy wrote in the article, which
he worked on for six months. ``If I were to come to
believe the opposite, then I would be morally obligated
to stop this work. I can now imagine that such a day
may come.'
Joy enjoys a level-headed reputation in the industry.
``Nobody is more phlegmatic than Bill,' said Stewart
Brand, an Internet pioneer. ``He is the adult in the
room.'
Joy is disturbed by a suite of advances. He views as
credible the prediction that by 2030, computers will be
a million times more powerful than they are today. He
respects the possibility that robots may exceed
humans in intelligence, while being able to replicate
themselves.
INEXPENSIVE SMART MACHINES
He points to nanotechnology -- the emerging science
that attempts to create any desired object on an
atom-by-atom basis -- and agrees that it has the
potential to allow inexpensive production of smart
machines so small they could fit inside a blood vessel.
Genetic technology, meanwhile, is inexorably
generating the power to create new forms of life that
could reproduce.
What deeply worries him is that these technologies
collectively create the ability to unleash
self-replicating, mutating, mechanical or biological
plagues. These would be ``a replication attack in the
physical world' comparable to the replication attack in
the virtual world that recently caused the shutdowns
of major commercial Web sites.
``If you can let something loose that can make more
copies of itself,' Joy said in a telephone interview, ``it
is very difficult to recall. It is as easy as eradicating all
the mosquitoes: They are everywhere and make more
of themselves. If attacked, they mutate and become
immune. . . . That creates the possibility of
empowering individuals for extreme evil. If we don't
do anything, the risk is very high of one crazy person
doing something very bad.'
What further concerns him is the huge profits from
any single advance that may seem beneficial in itself.
``It is always hard to see the bigger impact while you
are in the vortex of a change,' Joy wrote. ``We have
long been driven by the overarching desire to know
that is the nature of science's quest, not stopping to
notice that the progress to newer and more powerful
technologies can take on a life of its own.'
Finally, he argues, this threat to humanity is much
greater than that of nuclear weapons because those
are hard to build. By contrast, he says, these new
technologies are not hard to come by. Therefore, he
reasons, the problem will not be ``rogue states, but
rogue individuals.'
Joy acknowledges that to some people, this may all
sound like science fiction. ``After Y2K didn't happen,'
he said, ``some people will feel free to dismiss this,
saying everything will work out.'
Joy is less clear on how such a scenario could be
prevented. When asked how he personally would stop
this progression, he stumbled. ``Sun has always
struggled with being an ethical innovator,' he said.
``We are tool builders. I'm trailing off here.' |
