[version_five]

Something that can escape it's programming, not just master what it's been given. I expect there are mental gymnastics to explain why chatgpt already fits this in some technical sense, hopefully people see what I'm getting at.

[nullsense]

I definitely see what you're getting at. I think this is the crux of the alignment problem in general. Though, in a deeper sense, what is the property that allows that to be possible?

I suspect it at least involves the combination of being able to continuously learn in response to novel stimuli and developing the goal of self-preservation.