Creating a New Intelligent Species: Choices and Responsibilities for AI Designers
Eliezer Yudkowsky
page 5 of 5
I, Eliezer Yudkowsky, am the son of human parents; but my parents did not create a new intelligent species in creating me. If you create a new intelligent species, even though that species has 
but a single member, then that is not just a child of the programmers, it is a new descendant of the family that began with Homo sapiens. It is a child of humankind. That is not something to be undertaken lightly. It is not a trivial art to create a species and person that lives a life worth living. AI researchers have had enough trouble creating intelligence at all. Nor is it ethical to make something exactly resembling a human, if you have other options. Naturally, darkness is carved into our genes by eons of blood, death and evolution We have an obligation to do better by our children than we were done by.
What makes a child of humankind? It is an impossible question to answer exactly, but three defining attributes exist as warning signs. They are not sufficient to make a person, let alone a happy person. They are to be used more as a guide when making a computer program; if one is possessive of these three attributes, then you are trespassing on people territory. First, do not build an AI that starts talking about the mystery of conscious experience and its sense of selfhood. Second, do not build an AI that wants public recognition of personhood and resents social exclusion inherently. Finally, do not build an AI that has a pleasure/pain re-enforcement and a complex powerful self model because at that point human beings are going to start empathizing with the AI. That is, do not do these things unless you are willing to tackle the full burden of responsibility of creating an intelligent species and a person that lives a life worth living.
Maybe it will turn out that there is no way to create a powerful intelligence without personhood. Natural selection, for example, is a powerful optimization process that creates complex designs, and yet natural selection is definitely not a person, nor is evolution even close enough to a person for the analogy to make sense. My hope is that I can create a powerful optimization process without creating a person. It will not be easy to understand the confusing things well enough to intentionally not build a person, if that is even possible. Nonetheless, it would take a far higher order of art to understand those things well enough to make a person that lives a life worth living. If it is not ethical to build the human, what is it ethical to build? What is right and proper in the way of creating a new intelligent species?
Hopefully, the project that first creates Artificial Intelligence will have some reasonably smart people who are passionate about AI. This does not necessarily mean that their passion will express itself correctly, but at least they will care genuinely about AI. We, the human species, should try to show total grace in this challenge. We have a responsibility to do better by our children than we were done by.
Eliezer S. Yudkowsky is a Research Fellow at the Singularity Institute for Artificial Intelligence. Yudkowsky happened to pick Vernor Vinge's True Names off a library shelf at the age of sixteen, and has specialized directly in the Singularity ever since. Yudkowsky is best known for his activist stance on the Singularity; that the Singularity will enormously benefit humanity, and that we should therefore try to accelerate the Singularity. Yudkowsky's professional work focuses on Artificial Intelligence designs which enable self-understanding, self-modification, and recursive self-improvement ("seed AI"); and on Artificial Intelligence architectures that enable the creation of sustainable and improveable benevolence ("Friendly AI").