| > I see no reason to believe that all AGI (or GAI, to use your acronym) entities will be identical I absolutely agree. There will be a Cambrian explosion of superintelligent forms. > The additional value that humans present would thus be a diversity factor. We already have a test for this. Parrots, octopus, whales and many other creatures are also diverse. But our problems are beyond their experience. Do we value their diversity as thinkers in our economy? We don't. (I am speaking in economic terms. We can deeply intensely highly "value" many things, but if we don't actually invest economic value in protecting or nuturing those things, our appreciation has little or no impact.) -- > It comes down to timing: if there's technology available for humans to become less ... biological That makes sence. The order things become viable makes all the difference. What I am seeing is that AI as software/minds is moving faster and faster. AI "bodies", i.e. robotic forms, are moving much slower but gaining steam. And our ability to augment biological forms is moving far far slower. All three areas are accelerating, but the disparity between them isn't shrinking - for now. But who knows? The next couple decades will be interesting! Maybe going fully artificial ("self-designed" might be a more meaningful description) will become trivial sooner than we think. :) -- > Lastly, I'm suspicious that GAI entities won't be purely economically motivated, because I don't see any reason that they'll be "purely" anything at all. I think they will be deeply motivated economically to care about ethics. Just as our genes and culture have (slowly) responded to the tremendous value of creating win-win situations. Also, our general curiosity about about many things that don't have immediate economic value, actually does drive unpredicted personal and society level economic advances over the long run. So I expect they will maintain interest in general and undirected learning. Similarly, our appreciation for seemingly non-economic shows of creativity (art, music, competitive games, mathematics for its own sake, etc.) can be thought of as the active form of curiosity. The development of novel and surprising artifacts and challenging competitions, may have value to AI's as well. Everything that is important to us became that way for some economic reason: specifically, survival energy economics. So many of our seemingly non-economic activities may continue to have analogous roles for future intelligences. |
> We don't.
I don't mean humans as they presently are, but humans as somewhat-novel GAI entities after upgrading (and humans as an ongoing source: likely there will be some humans that aren't interested in changing themselves in such ways, but who might change their mind over time and/or have children who see things differently).
> (I am speaking in economic terms. We can deeply intensely highly "value" many things, but if we don't actually invest economic value in protecting or nuturing those things, our appreciation has little or no impact.)
I don't think this is true. For example, suppose you deeply value a bunch of things that you don't think have much economic significance, but economic significance is the most important thing to your perspective / philosophy, so you don't take care of the things that you deeply value. This does have a significant impact: you become sad, you become depressed, you become less economically productive. So then you say, oh, I'll just take care of those other things I care about, and then I can be economically productive! So you make a few changes but then find yourself feeling down again not long afterwards because economic value is the cornerstone of how you think about things. And then you realize that from an economic perspective, it's more beneficial economically to stick with a perspective that embraces your values as you discern them (and how those continue developing over time) rather than sticking with a perspective that puts economic value first. And then you are taking your first steps to freedom from the curse of your Bachelor's in Economics. ;)
Ethical GAIs also seem likely to me ... hopefully.