[strangescript]

Or all these properties are emergent from ever increasing compute. The underlying architecture doesn't need to fundamentally change. The roadmap is increasing compute. Agency is simply never ceasing input and resulting output, both internally and externally. Do you have Agency when you sleep? I still don't know what "extrapolate beyond their experience" means. Does Einstein discover relativity if he is never taught math? More to the point, why was he so brilliant in the first place. Why could he "extrapolate beyond their experience" better than others in his field?

We want AI to hoop jump something we don't even understand ourselves. The only empirical evidence we have is as we increase compute, the results get better.

[fnordpiglet]

I would note you absolutely have agency when you sleep - in fact a lot of the purpose of dreams is to simulate situations for you to practice in safely - that’s why you’re often replaying situations you’re worried about or things you commonly experience. Also lucid dreaming is still fully immersed dreaming but you have total agency and awareness. The key to lucid dreaming is just training yourself to not give up aware of the non dream reality control while dreaming.