[empiricus]

I apologize for the tone of my comment, but this is how I read your arguments (I was a little drunk at the time):

1. future AI cannot be infinitely intelligent, therefore AI is safe

But even with our level of intelligence, if we get serious we can eliminate all humans.

2. some smart ppl I know are peaceful

Do you think Putin is dumb?

3. smart ppl have different preferences than other ppl therefore AI is safe

Ironically this is the main doom argument from EY: it is difficult to make an AI that has the same values as us.

4. AI is competent enough to destroy everyone but is not able to tell fact from fiction

So are you willing to bet your life and the life of your loved ones on the certainty of these arguments?

[godelski]

  > I was a little drunk at the time

Honestly it still sounds like you are. You've still misread my comment and think I said there can't be bad smart people. I made no such argument, I argued that intelligence isn't related to goodness.

[bondarchuk]

If that was what you meant to say though, you've gotta admit that opening a paragraph with "The other weird assumption I hear is about how it'll just kill us all", and then spending the rest of the paragraph giving examples of the peacefulness of smart people, is not the most effective strategy of communicating that.

[godelski]

You were the one who interpreted "Here's examples of smart peaceful people" as "smart == peaceful". I was never attempting to make such a claim and did say that. The whole thread is about bad assumptions and bad logic.

  > is not the most effective strategy of communicating that.

The difficulty of talking on the internet is you can't know your audience and your audience is everybody. Yes, this should make us more aware about how we communicate but it also means we need to be more aware how we interpret. The problem was created because you made bad assumptions about what I was trying to communicate. There are multiple ways to interpret what I said, I'm not denying that, it'd be silly to because this is true for ANY thing you say. But the clues are there to get everything I said and when I apologize and try to clarify do you go back and reread what I wrote with the new understanding or do you just pull from memory what I wrote? Probably isn't good to do the latter because clearly it was misinterpreted the first time, right? Even if that is entirely my fault and not yours. That's why I'm telling you to reread. Because

  >>>> So we can't conclude that greater intelligence results in greater malice.

Is not equivalent to

  >>> assuming that intelligence will necessarily and inherently lead to (good) morality

We can see that this is incorrect with a silly example. Suppose someone says "All apples are red" and then someone says "but look at this apple, it is green. In fact, most apples are green." Forget the truthiness of this claim and focus on the logic. Did I claim that red apples don't exist? Did I say that only green exists? Did I forget about yellow, pink, or white ones? No! Yet this is the same logic pattern as above. You will not find the sentence "all smart people are good" (all apples are green).

Let's rewrite your comment with apples

  > If that was what you meant to say though, you've gotta admit that opening a paragraph with "The other weird assumption I hear is about how all apples are red", and then spending the rest of the paragraph giving examples of different types of green apples, is not the most effective strategy of communicating that.
 

Do you agree with your conclusion now? We only changed the subject, the logic is in tact. So, how about them apples?

And forgive my tone, but both you and empiricus are double commenting and so I'm repeating myself. You're also saying very similar things, we don't need to fracture a conversation and repeat. We can just talk human to human.

[bondarchuk]

I think the big difference between our views is that you are taking the rationalist argument to be "from intelligence follows malice, therefore it will want to kill us all" whereas I take it to be "from intelligence follows great capability and no morality, therefore it may or may not kill us uncaringly in pursuit of other goals".

[godelski]

  > you are taking the rationalist argument to be

I think they say P(doom) is high number[0]. Or in other words, AGI is likely to kill us. I interpret this as "if we make a really intelligent machine it is very likely to kill us all." My interpretation is mainly biased on them saying "if we build a really intelligent machine, it is very likely to kill us all."

Yud literally wrote a book titled "If Anyone Builds It, Everyone Dies."[1] There's not much room for ambiguity here...

[0] Yud is on the record saying at least 95% https://pauseai.info/pdoom He also said anyone with a higher P(doom) than him is crazy so I think that says a lot...

[1] https://ifanyonebuildsit.com/

[bondarchuk]

Yes, I agree they are saying it is likely going to kill us all. My interpretation is consistent with that, and so is yours. The difference is in why/how it will kill us; you sound to me like you think the rationalist position is that from intelligence follows malice, and therefore it will kill us. I think that's a wrong interpretation of their views.