[sharpener]

Efficient proprioception is useless if key behavioural incentives are misaligned with key aspects of the environment.

Humans are quite good at creating collective organisations that are delusional, even though some of those organisations have excellent information about the state of their environment, because the invented incentive structures for action favour simplistic short term "maximisation" of fetishised behaviours instead of optimisation for (even) the medium term healthy future. Resource allocation is still looking frighteningly suboptimal right now, and simply empowering that with a "smart" central AI bot following the same patterns probably won't improve much.

I'm skeptical about the human race getting past this. Humans want a lot of stuff, and are prepared to form huge organisations to get that stuff, without really thinking through whether they really need it, what the other impacts will be, or why they are even doing what they do.

What is the human race trying to be?

Why?

How does the pattern of incentives get aligned?

It seems like we only have the weakest responses to these questions right now.

[threatripper]

Those are important questions we seem to desperately avoid as a society. Things are happening despite not having those answers.

My guess is that humans will be sidelined as AI is developing exponentially. There probably won't be a big war human vs machine instead we will probably see humans being turned into some kind of zoo animals as we already are if you look from far away. Then the AI will do its thing in space while we sit here happily staring into the stars.