Creating a New Intelligent Species: Choices and Responsibilities for AI Designers

Eliezer Yudkowsky

page 3 of 5

The human brain is full of complicated machinery and human universals that are complex adaptations crafted by natural selection. These are easy to accept as an abstract fact, but hard to remember in particular cases.

Suppose I pointed to a particular piece of neural machinery or neural circuitry and asked you whether it was more natural for this piece of circuitry to project to the contra lateral insula or nucleus accumbens? The way that question was phrased, there is no obvious answer. Nerve fibers can lead anywhere depending on how the genes wire them. As it so happens, the contra lateral insula is one of many brain areas involved with pain and the nucleus accumbens is one of many brain areas involved in pleasure. If I asked you whether it is more natural for a hug from a loved one to feel pleasurable or painful, you have a ready answer for that. But the brain did not magically wire itself that way.

Natural selection produced a particular brain design, wired one way instead of the other. It takes a conscious effort to realize that the brain is full of machinery working behind the scenes. It is clear enough why evolution gave you a brain such that a hug from your loved one feels nice instead of awful. Yet, and this is the critical point, when you build an artificial intelligence, you as the programmer would choose for the AI those things that evolution chose for us.

The programmer must decide what kind of emotions the AI will have. When will the AI feel those emotions, at what intensity, and for how long? What brings pleasure? What brings pain? Or maybe the programmer will build a different kind of mind that does not feel pleasure or pain at all. Everything is up for grabs. With that comes the ability to commit brand new crimes, crimes for which there are no names yet. Is it a sin to create a mind that feels a hug from a loved one as pain? If you rewire everything that goes from the contra-lateral insula to the nucleus accumbens, and vice versa, without changing any of the other neural areas involved in pleasure and pain, what happens to that mind? What happens to a child that is raised like that? The answer is not clear, but most would agree that anyone who does such a thing to any child, human or otherwise, deserves to go to jail.

In the case of Artificial Intelligence, we are not talking about damaging a brain that would otherwise be healthy. You cannot mess with an AI's nature because an AI does not have a pre-existing nature; it is all up to the programmer. We are not talking about the crime of assault, of hitting someone on the head and causing brain damage. We are talking about the crime of designing, and then creating a broken soul.

One of the major surprises to emerge from research in hedonic psychology (the science of happiness) is that humans have a happiness set point. No matter what happens to us, we soon adjust back to that set point. There are very few things that have been shown to have a long-term effect on human happiness. Neither winning the lottery, nor losing limbs, is on the list. The only good predictor of individual variance in long-term happiness is how happy our parents are.

1 2 3 4 5 next page>