[pixl97]

People seem to focus on the AI we have now in these threads, which I guess is a whole lot easier than the speculative alignment guessing on something that could end up being a whole lot smarter than you, and be able to input far more types of information than you ever will.

Personally I don't see anyway to make something that is super human and aligned outside of its own choice. How to make something that is both beyond us, and have it come to the conclusion not to extinct us will be interesting enough as your example above shows we are real jerks to each other already.

[TeMPOraL]

That's what "alignment" used to mean until about a year ago; the term has since been hijacked to extend to making LLMs polite and obedient. This leads to confusion and people asking "what's the big deal with the 'alignment' thing?". The big deal is with "avoiding getting casually extincted by a powerful enough AI" kind of alignment. The "reliably preventing LLMs from saying undesired things" is much lesser issue (though probably a small part of the big problem).