[goatlover]

That doesn't address the alignment problem. As Yudkowski has pointed out, the space of possible minds is vast. Humans only occupy a small area. The models were are designing are not animal/biological minds. They don't have the same evolutionary drives. We're creating minds in a different part of the mind space, and there's a good chance they will figure out solutions that are not beneficial to us.

[MrScruff]

Absolutely, I would just addressing the ‘it’s just statistical’ argument. The vast majority of human behaviour is learned. We’re all implicitly ‘saying what seems right to achieve some underlying fitness function’ all of the time. That’s what fashion is about. But yes, definitely no reason to assume an AI will think like us underneath it all.