[nullsense]

I definitely see what you're getting at. I think this is the crux of the alignment problem in general. Though, in a deeper sense, what is the property that allows that to be possible?

I suspect it at least involves the combination of being able to continuously learn in response to novel stimuli and developing the goal of self-preservation.