[laretluval]

> Like all lifeforms, our goal should be to hand our future to better and better offspring.

How did you determine the moral purpose of all life? I say our goal should be to hand our future off to our own biological offspring, the members of our species, as evolution has optimized us to do. What makes your proposed purpose more correct than mine?

[staunch]

It is (kind of) our biological offspring, we'll just create it using our brain fluids instead of our reproductive fluids ;-)

Your limited primate brain is the reason you want primate offspring. This bias is in our DNA and part of our evolution. It's a limitation that your super AI offspring would not have.

[laretluval]

Maybe our primate brains are primitive, or maybe they are infected with a kind of disease that will result in the eradication of our species. Certainly if we observed self-extinctive behavior in the wild, we would classify it as a behavioral anomaly and perhaps even try to fix it, like we do with the pandas who refuse to reproduce.

But more generally, of the two ways of seeing it, who's to say which is right? When we disagree, to what can you appeal?

My point is, these moral questions are completely arbitrary. There is no grand cosmic objective function for life that is objectively true. Using these quasi-religious arguments about the "purpose of life" to guide AI safety research seems extremely irresponsible.

[goatlover]

But why the preference for super AI and not super cockroaches? I'm thinking our primate DNA biases us toward intelligence over insectile qualities.

Personally, I think super octopuses are the way to go. They can make better use of both land and sea, and all the cool aliens look like that.