[sometimelurker]

providers do not want power-seeking LLMs. no one does. this (bad personality) is incentivized during training, especially RL, and is something they would rather not have. tell me, do you think training a power-seeking ASI is a good idea?

[dag100]

> providers do not want power-seeking LLMs

Sure they don't.

> do you think training a power-seeking ASI is a good idea?

Do you really think that the major AI firms think this is a bad idea? I think the whole drama people have around ASI taking over the planet is ridiculous and fantastical and, honestly, a way to distract from the real problems of ASI - that is, creating a rigidly hierarchical world with zero social mobility.

[sometimelurker]

'X-risk/S-risk' is a superset of 'hierarchical world with zero social mobility' and 'ASI taking over the planet is ridiculous'. x-risk=bad, smart enough power-seeking llms automatically lead to x-risk