Hacker News new | ask | show | jobs
by JohnFen 740 days ago
There may be a nonzero chance of developing a superintelligence (or even AGI in any meaningful sense), but it's so close to zero that it doesn't matter.

Particularly because if a superintelligence is actually created, it won't matter who did it. It won't be the servant of the creators.
2 comments
arisAlexis 739 days ago
You are extremely confident about what all 3 top AI labs and top 3 cited experts on the subject disagree with.

Should I repeat the common quote of the problem in this world is....

link
JohnFen 739 days ago
Yes, I am. I'm not certain, of course, which is why I acknowledge a nonzero chance of it.

There is not actually consensus about this amongst the experts in the field.

link
arisAlexis 738 days ago
As I said, there actually is. All top 3 scientists and all 3 top labs that have built the tech you are referring too. It's unanimous.

Sources: h-index of AI rankings LLM rankings

Everyone can look them up and what their stance is.

link
Jimmc414 740 days ago
>it's so close to zero that it doesn't matter.

How can you so confidently assert that? AI just needs to beat humans at AI research to start compounding it's growth in performance.

>It won't be the servant of the creators

Human's are very paranoid and competitive because we've had to be for tens of thousands of generations to survive. Super intelligent or not why would AI start ignoring instructions or develop a survival instinct? This line of thinking has been popular in science fiction but what is it based on? Just because something is smarter than us, it will kill us, as Eliezer Yudkowsky asserted "because it wants our atoms"? It seems like we are viewing AI through a human lens.

Superintelligence under human control is what scares the shit out of me. I see superintelligence as the only thing capable of saving humans from themselves.

link
JohnFen 739 days ago
> How can you so confidently assert that?

Because I've seen no strong reason to think otherwise. However, everyone -- myself included -- is just guessing here. I could certainly be wrong. As could the ones of the opposite opinion.

> AI just needs to beat humans at AI research to start compounding it's growth in performance.

That's assuming that it's possible to create an AGI at all given our current direction. I seriously doubt that it is. Time will tell, though.

link
montieferg 740 days ago
love this take.
link