[carapace]

I wish I could upvote this remark twice (or more.)

This is pretty much the only important concept for figuring out how we will use this tech politically.

Because it takes genius-level intelligence to be able to figure out whether you're just telling yourself what you want to hear, and incredibly rare responsibility to remember to [keep on trying to] do so, individuals and tiny groups may be able to use AI for these sorts of things, but large groups, municipalities, states, corps, etc. never will.

The systems we can understand and manage as a group are vastly simpler than those which you can understand and manage as an individual.