[slenocchio]

Listen to Eliezer Yudkowsky's arguments. Perhaps it looks silly to you but there are actually _extremely_ good reasons to be worried about AI progress without figuring out alignment first. What would happen if we gave humanity the internet before we had cryptography?

On a side note, it's troubling how many hyper intelligent and successful people in tech are happy to denigrate the opposition in bad faith.

[vivekd]

I don't understand how alignment is even an issue with a large language model? It seems to me like we are a very long away from developing the kind of AI where alignment even becomes an issue. We seem to be focusing on the "AI" moniker without considering what this technology actually is, which is basically just a computer running a statistical algorithm on a large collection of sentences

[slenocchio]

I'll try summarizing Yudkowsky's main point as I understand it (however you should read him directly if you find me unconvincing).

Obviously the current version of ChatGPT and LLMs of today are not world threatening. The problem is we are blindly pushing forward and __we don't know__ what to expect. You say you think we are very far away from a point where alignment becomes an issue, yet _no one_ predicted AI would've come this far so quickly a couple years ago.

Natural selection is an algorithm that produced humans, which do all sorts of weird things that seem contrary and irrelevant to the goal of that algorithm (use birth control, build space ships)

With AI, instead of natural selection, we have gradient descent (a more sophisticated algorithm; it takes into account the derivative) and we are blindly throwing more resources into this algorithm hoping it will turn out ok.

Normally in science we can screw up and try again. But with AI it's different. If you create a super-intelligent AGI not aligned with humanity you are playing Russian roulette with life on Earth with 5/6 chambers loaded.

Yudkowsky's primary concern is literally just "can we build AI with some degree of certainty we won't destroy all of humanity?" which seems laughable, but genuinely, I haven't heard one person go up against him that clearly didn't expose themself as less thoughtful and intelligent about the matter _by far_ than Yudkowsky. I wish I heard an argument against him that was convincing.

Besides that concern, of course we have the more tangible, perhaps more probable concern of "bad actors exist in the world, what happens when we democratize (or simply create) the most powerful tool ever created?"

What happens when governments, corporations, and individuals can use LLMs to turn every communication channel (Hacker News, TV, Twitter, etcetera) into 99% AI noise with superhuman-quality propaganda, misinformation and advertising for their own self-interest? I'm sure there's a million other ways AI could be used by a self-interested minority to the detriment of humanity.

[gumballindie]

> Normally in science we can screw up and try again. But with AI it's different. If you create a super-intelligent AGI not aligned with humanity you are playing Russian roulette with life on Earth with 5/6 chambers loaded.

Aight. Lets leave it as it is. The disconnect from reality is real and concerning.

[gumballindie]

> internet before we had cryptography?

I’m sorry what? The internet has been clear text for the most part.

[slenocchio]

We can communicate securely on the internet because of cryptography (HTTPS vs. HTTP). If cryptography didn't exist any signal you ever sent over the internet could be observed and used maliciously against you (payment info, passwords, etcetera).

[argiopetech]

The WWW (1989) ran for 5 years before SSL 1.0 (1994).

If cryptography were never invented, we would use the internet like we used analog telephones (i.e., under the assumption our communication could be easily intercepted) and not pass privileged information over it. Otherwise, I imagine it would be a non-issue.

Given changing laws on cryptography (i.e., state-required backdoors & etc.), we may be back in that mindset before too long.

[slenocchio]

Appreciate your comment! You're right, the analogy isn't perfect. I hope the sentiment of the analogy comes through though.

"Otherwise, I imagine it would be a non-issue." Don't understand how you can say that though.

[argiopetech]

Re. "non-issue": if you don't send privileged information across an insecure link, that information can't be intercepted on that link. If the WWW were insecure, we wouldn't use it to do things that require security. We'd use the systems the internet replaced instead (primarily interaction with real people in brick-and-mortar settings).

[gumballindie]

But there we go again - asking people to bend reality to match your narrative.

[slenocchio]

Not asking anyone to bend reality. Most analogies/metaphors aren't perfect and require good faith by the listener to see the truth in them.

"A mind is like a parachute; it doesn't work if it is not open." Some people can hear this and understand what Frank Zappa means. Others (perhaps like yourself) will sit there and whine "bUt wHaT aBoUt tHiS cAsE wHeRe iT sHoulDn't be oPeN!!"

[mikeravkine]

1997 era internet would have scared the shit out of you.