[danaris]

...do....do you think that's a gotcha?

Yes, if we had considered ethics in the past, we wouldn't have committed bloody genocide on the Native Americans. We would probably have tried to open up mutually beneficial trade with them. (Sadly, we probably would still have caused the deaths of millions of them simply due to pathogens we didn't know we were carrying when we met them.) Do you think that would have made things worse now?

> In my moral framework, the worst (i.e. greediest, most selfish, anti-social) people have been winning over the last 20-40 years.

This sounds an awful lot like a "might makes right" attitude: "Sure, they're awful people, but they're winning, so we should adhere to their values!"

[jsrozner]

I realize it might have come across like I was advocating something akin to nihilism, or as you put, "might makes right." It's the opposite. I'm saying that so far, caring about ethics--especially over the past couple decades--hasn't mattered or has been even counterproductive to one's economic station. And relatedly, the people who have cared least about ethics have won, which in turns has made caring about ethics even less economically viable.

Deciding "what is worth doing" requires adjudicating values, and this does not have a mathematical solution.

The development of AI is a bit like giving everyone an atomic weapon, and few people have the ethics to use it responsibly (for some technologies, there does not even exist an ethical use).

The author suggests that the people who can tell "what's worth doing" (roughly, what's right) are the ones who will succeed in the coming era. I actually, unfortunately, think it's the opposite. The people who most ruthlessly steal and deploy AI for their own gains are likely to "succeed," where "succeed" is defined by accruing greater power. Those of us who are worried about doing what is right are likely to move more slowly and carefully and to thus "get left behind."

[alpha_squared]

I don't think the parent is advocating adhering to their values, but instead just talking about the relativism of ethics and morality.

Society broadly cannot function without a shared set of morals (e.g. stealing sows discord which leads to anarchy, so we outlaw stealing). However, it can get quite complex and so there are infinitely many shades of grays. The base principal would still stand and you do your best to adhere to it in order to have a stable society.

[jsrozner]

Right I'm definitely not advocating that one should adhere to the values of the winners of the last 20-40 years.

I'm saying that if you're winning today (as measured in money/power), then by most objective notions of morality, you're probably a crappy, antisocial, person who is greedy and exploitative.

At some level it's almost definitional: if you share the material wealth you have, you'll never become that wealthy. If you don't consume a bunch of resources for the sake of your own consumption or to produce products that others consume, then you're also not going to make a lot of money.

[pixl97]

No! Not their values, my values because my values are the best.

So ya, it's a hard problem.

[danaris]

Nah, moral relativism is bullshit.

Sure, I can acknowledge that there may be values out there that are better than my values. That doesn't mean I have to say that people whose values are antithetical to mine—whose values are essentially "I can, should, and will take anything I want, no matter the cost to other people"—are just as good.

[pixl97]

And I mean I agree with you, but must acknowledge that they may not agree with you and I and are willing to bury you over it.

So as I said, a hard problem.