How We Can Manage Our Way Through the Intertwined Promise and Peril of Accelerating Change
Ray Kurzweil
page 14 of 15
That is the threshold we are on now. The next major challenge will be nanotechnology. There is a lot of effort now to say that 
nanotechnology manufacturing does not require self-replication. That is basically true, but the lesson from that is not that self-replication is no longer a concern, that grey goo was not a real specter. Grey goo is a possibility. In fact, manufacturing that does not use self-replication still does have self-replication hidden within the system. Techniques that use things like the broadcast architecture, where an entity does not have all the codes needed to self-replicate itself are a good idea. That inherently will make nanotechnology safer than biology. In biology, as a cell replicates, it has all of the replication codes within it. It does not have to go to some centralized server. But there are ways of defeating this broadcast architecture if you are determined to defeat them.
I do not think the main danger is actually from accidents. I think it is very hard to just accidentally put together something that is deadly at a massive scale. Certainly, accidents can happen on a small scale. For example, it is not easy to just sort of accidentally put together an atomic bomb. Any of these real dangers require very exquisite engineering directed at a destructive goal. But we cannot assume that people will not in fact do that. Recent history shows that people will do that for whatever reason. We ultimately will need a nanotechnology immune system.
Does a nanotechnology immune system need to have self-replication for the blue goo, that is to say the good nanobots to keep up the with grey goo or the bad nanobots? I had an interesting debate in writing this book with Rob Freitas about this very issue, who has a proposal for a nanotechnology immune system that does not require self replication, where you detect the nanobots, you quickly decide on a design of an antidote, some nano system that would destroy it, kind of like RNA interference in the biological world, and then use nanomanufacturing to put out large numbers of them.
I think, without going through the details, a scenario like that will work in the early stages. Yet any particular system that you put in place is not going to last indefinitely. The technology is going to get more sophisticated. The nanobots themselves will get more intelligent. Ultimately, when you have achieved a certain level of intelligence in the nanosystem itself, a static non-replicating immune system is not going to work. That is the lesson that biology "learned." It evolved an immune system that does have self-replication in it. This immune system itself can represent a risk. Bill Joy points out that the immune system itself could turn on us.
And that, of course, is true in the biological world as well. We have auto-immune diseases, but that is not a reason to not have an immune system. We would not last very long without one. We will need an immune system. Ultimately, the early ones may not need self-replication or the self-replication can be hidden or it can use the broadcast architecture. Yet ultimately, we will need an immune system that does have self-replication.