| I'll try summarizing Yudkowsky's main point as I understand it (however you should read him directly if you find me unconvincing). Obviously the current version of ChatGPT and LLMs of today are not world threatening. The problem is we are blindly pushing forward and __we don't know__ what to expect. You say you think we are very far away from a point where alignment becomes an issue, yet _no one_ predicted AI would've come this far so quickly a couple years ago. Natural selection is an algorithm that produced humans, which do all sorts of weird things that seem contrary and irrelevant to the goal of that algorithm (use birth control, build space ships) With AI, instead of natural selection, we have gradient descent (a more sophisticated algorithm; it takes into account the derivative) and we are blindly throwing more resources into this algorithm hoping it will turn out ok. Normally in science we can screw up and try again. But with AI it's different. If you create a super-intelligent AGI not aligned with humanity you are playing Russian roulette with life on Earth with 5/6 chambers loaded. Yudkowsky's primary concern is literally just "can we build AI with some degree of certainty we won't destroy all of humanity?" which seems laughable, but genuinely, I haven't heard one person go up against him that clearly didn't expose themself as less thoughtful and intelligent about the matter _by far_ than Yudkowsky. I wish I heard an argument against him that was convincing. Besides that concern, of course we have the more tangible, perhaps more probable concern of "bad actors exist in the world, what happens when we democratize (or simply create) the most powerful tool ever created?" What happens when governments, corporations, and individuals can use LLMs to turn every communication channel (Hacker News, TV, Twitter, etcetera) into 99% AI noise with superhuman-quality propaganda, misinformation and advertising for their own self-interest? I'm sure there's a million other ways AI could be used by a self-interested minority to the detriment of humanity. |
Aight. Lets leave it as it is. The disconnect from reality is real and concerning.