[jsrozner]

I realize it might have come across like I was advocating something akin to nihilism, or as you put, "might makes right." It's the opposite. I'm saying that so far, caring about ethics--especially over the past couple decades--hasn't mattered or has been even counterproductive to one's economic station. And relatedly, the people who have cared least about ethics have won, which in turns has made caring about ethics even less economically viable.

Deciding "what is worth doing" requires adjudicating values, and this does not have a mathematical solution.

The development of AI is a bit like giving everyone an atomic weapon, and few people have the ethics to use it responsibly (for some technologies, there does not even exist an ethical use).

The author suggests that the people who can tell "what's worth doing" (roughly, what's right) are the ones who will succeed in the coming era. I actually, unfortunately, think it's the opposite. The people who most ruthlessly steal and deploy AI for their own gains are likely to "succeed," where "succeed" is defined by accruing greater power. Those of us who are worried about doing what is right are likely to move more slowly and carefully and to thus "get left behind."