Ethics for Machines

J. Storrs Hall, Ph.D.

Page 3 of 3


Image 11: The Horizon Effect

Here is another invariant. This was noticed Sort of instantly and painfully by the early chess programs that actually did use the rational architecture. If you have a short planning horizon, so you're searching the game in terms of who is going to do what move next, but only go so far, then you might take a pawn with your queen not realizing that somebody is going to take your queen the next move because you didn't look at the next move. That was called the Horizon Effect. Actual test programs have all sorts of tricks and techniques for getting around it, but the basic point is that the longer your planning horizon is, the better off you'll be.

In fact, if you're living in an environment of other intelligent or super-intelligent creatures, the more likely anything that you do that is cheating or underhanded or whatnot is going to be discovered and come back to haunt you, the probability of that goes up.

"The more intelligent AIs or other intelligent minds that are in your environment, the more likely a long planning horizon is to make you be honest."

It doesn't actually have to be quite so much the ability to predict specifically what's going to happen. If you have general rules such as, honesty is the best policy, this can help you predict in a general way what results your actions will have even if you don't know  the results of any particular action, if the rule turns out to be a reasonably good average over all the situations that you're going to meet. Rules of that kind could become part of the world model in probabilistic terms, and therefore, you would expect that the AI that had a tendency to try and lengthen or broaden its planning horizons would retain that property over evolutionary change.


Image 12: Invariant Traits

So let's sum up the traits that I think will be invariant. Before I do that let me point out that this is a handful of things that I picked out that seem to be traits that are going to be invariant in evolving AIs but they don’t have to be the only ones. The really important part of this talk is the notion that you have to find invariants if you are going to design the AI now whose activities you can continue to trust in the future. You start them out with good versions of all the invariants that are going to contribute to proper behavior.

So we have the love of knowledge. We have self-interest, and self-interest is actually not necessarily bad.  People like to think of self-interest as bad sometimes, but self-interest means at least that you're going to worry about what's going to happen to you and if other people are going to get back at you for something, then you will have a reason not to just run out and be a berserker, for example.

"If you understand the evolutionary dynamics of the moral ladder, you're much more likely to try and be on that ladder."

If you're capable of guarantee-able trustworthiness, the AIs will, I think, be much more likely to be able to form quickly set-up, cooperating communities and produce projects and efforts without the sort of mutual distrust and problems that humans have.

The long planning horizon is essentially another form of an increased understanding of the world and an improvement of the world model in the architecture.


Image 13: The Moral Machine

If we start an AI with these traits, then Because they are invariants in the evolutionary process, I think they are very likely be conserved in future AIs, even though they ultimately come to Understand things more deeply than we do and not only their understanding but their motivations get changed beyond current recognition.

I think that it's quite reasonable to expect that in 50 years' time we will live in a world where AIs are essentially the only really moral creatures creatures and we are still a few rungs down the ladder, and essentially we will be having to look up to them and learn our morals at their knees, so to speak..

Bio

Josh Storrs-Hall, Ph.D.

Josh has been in the forefront of molecular nanotechnology for over a decade. A Research Fellow with the Institute for Molecular Manufacturing since 1998, Josh brings a background in parallel processor architectures, AI, and robotics to the study of self-replication at the nano level and its extrapolation for the construction of macroscopic machines. Josh received a patent for optimized computer design in 2002 and has written numerous articles and spoken at nanotechnology conferences since the mid-90s. He is also the author of the new book Nanofuture, What’s Next for Nanotechnology. Josh received his masters and doctoral degrees in computer science from Rutgers University in 1994.

1 2 3 <Back to Issue Contents