[DeonRob]

Yeah, but you don’t actually want intelligence — you want consistency. And LLMs are great at giving you the next word or phrase based on the probability that you truly believe a prompt and its context.

Which is exactly what you want. It’s like if you ask a group of five people what they want to eat, knowing they’re all vegans. If you wanted a fair decision-maker, you’d hope they pick something close — vegan and good. An AI will always give that answer in a boring and predictable way. But a human might take outside context into account or selfishly pick a non-vegan restaurant — not because it’s consistent, but due to any number of random outside factors.

[Zigurd]

An LLM might very well pick undesirable outcomes because all it's doing is remixing human knowledge which has led to plenty of undesirable outcomes.

LLM powered governance seems like a janky version of luxury communism.

[rbanffy]

But, people, I guaranteed fairness and wisdom. Just trust my model, OK? It's for your own good.

I give you my word.

[DeonRob]

Issue is "might" is better than the guaranteed corruption of socialist system at scale. You can adjust the AI, humans we know are to flawed to lead a altruistic movement or make constant selfless decisions

[Zigurd]

Debatable. E.g. "MechaHitler."