[miguelrochefort]

I have just read your link.

I'm one of those people that don't believe humans to be special and would happily see them being replaced by what made them obsolete.

I don't believe in Friendly AI. Yudkowsky's thesis is flawed. We should be replaced.

[TeMPOraL]

You don't like humanity, do you? :). Is your disagreement with FAI just about that we don't deserve to survive?

Also, I referred you to Meditations on Moloch not because of FAI, but because of the discussion on coordination problems. What do you think about the article's view of that?

[miguelrochefort]

I have no reason to dislike humans. They're the best thing we have today. That doesn't mean something better won't come to replace them.

Friendly AI, by definition, is not true AI. It won't lead to singularity. It's worthless.

What I want is for the universe to evolve as efficiently as possible. We shouldn't slow down progress just because we want to survive. The only "we" is the universe. Selfless we should be.

I'm not sure what solution he's offering to coordination problems. He speaks of consensus and monarchy. He criticizes classical capitalism.

I feel like the capitalist model would be good if incentives were adequate. Money and personal wealth aren't adequate motivations. Selfless capitalism, where systems are optimized toward making the universe better. A system where we all adopt god's-eye-view. That's what I was referring to with my universal absolute moral truth.

Those entities who don't exist for the greater good will be naturally selected out. The fittest will be the most righteous. We need a system to measure righteousness. I'm working on just that.

TL;DR: We can coordinate through the universal absolute moral truth. We just need better tools to accurately grasp it.

[TeMPOraL]

I see. I guess you're more selfless than I can ever dream to be - as long as it seems that we're alone in the universe, I want humans to shape and rule it :).

What you say would be a good solution, if we could actually shape incentives like that (and avoid the issue of incentivized incentivizers - an example of which are companies influencing governments to make laws better for those companies). You said that you're working on a part of this problem, on "a system to measure righteousness". This is something I think we could really use, at the very least to express the concept of "a douchebag" within legal system[0]. Can you share some details about your work? I'm interested.

--

[0] - Our legal systems try to stick to objective facts and avoid the topic of intent. Understandably so - without a way to read people's minds it turns into walking through a minefield. We can handle quite a lot this way, but in recent times a big part of suffering in this world is being created legally, by people with obviously selfish and malicious intent. We would want to discourage and/or punish such behaviour, but we can't because technically they aren't breaking any laws, even though everyone knows they are douchebags.