[maroonblazer]

Philosophers call this ability to predict another agent's behavior "theory of mind". Presumably it would be impossible to have a theory of mind - a theory about what another agent will do - without first being aware of your own.

[KKPMW]

To me it seems like you can have a state of mind, and use that state to predict the state of mind of others without being _aware_ of your state of mind at the same time - that is, without being conscious.

[ncmncm]

In principle, yes. In practice, it would be much harder to rig a system that can be aware of others' states of mind without being aware of its own. Nature never does things the hard way when the easy way works. Consciousness is the easy way.

[everfree]

Computers can make predictions about conscious agents and even revise their predictions based on past data, a counterpoint to the idea that agents cannot be predicted without theory of mind.

Unless I'm showing my ignorance here, which is quite likely.

[thechao]

I think it’s the other way around: we evolved the ability to build a theory of mind (to improve our social standing), and later applied that theory-of-mind to ourselves, greatly amplifying & reifying consciousness.