[kelseyfrog]

> So it's not at all clear how consciousness arises, and thus not clear that e.g. GPT-7 "cannot" be conscious.

Arguments from position that there is an essential human characteristic that AI cannot capture always leaves me wanting. It seems almost like their arguing from a position of ontological impossibility when I read their responses.

On one had we have a few simple classical physics equations which describe how matter behaves, and brains being matter, should follow these rules. On the other, we have universal function approximators - models which given enough resources can approximate any function. Theoretically it seems like one can be arbitrarily approximated by the other.

That's why dismissals feel like an argument from consequence. As one gap closes, another gap emerges. In the same way that science creates a god-of-the-gaps - incremental advances in science make god retreat into ever smaller and obscure gaps, advancements in AI create a consciousness-of-the-gaps where the justifications for consciousness retreat into ever smaller and obscure gaps. Embodiment is the newest of these gaps and once embodiment occurs, another gap will be discovered and retreated to.

Unfortunately, consciousness and sentience are social categories, not scientific categories - like countries, they gain legitimacy when enough agents say they're legitimate - and just like the gaps, this trend isn't stopping.