[monkeycantype]

Until we have a testable, falsifiable thesis of how consciousness forms in meat, it is rash to exclude that consciousness could arise from linear algebra. Our study of the brain has revealed an enormous amount about how our anatomy processes information, but nothing of substance on the relationship between matter and consciousness. The software and data of an operating LLM is not purely abstract, it has a physical embodiment as circuits and electrons. Until we understand how matter is connected to consciousness, we also cannot know whether the arrangements and movements of electrons meet the criteria for forming consciousness.

[layer8]

That’s largely a different topic from the article. Many people perfectly agree that consciousness can arise from computation, but don’t believe that current AI is anywhere near that, and also don’t believe that “thinking” requires consciousness (though if a mind is conscious, that certainly will affect its thinking).

[monkeycantype]

yes I agree it's not the angle of the article, but it is my entry point into the idea/concern/unanswered question at the end of the article “My worry is not that these models are similar to us. It’s that we are similar to these models.” - that the enormous difference in the medium and mechanics or our minds and llm's might not be that important.

before i go any further, let me first reference The Dude:

     - "this is just like, my opinion man."

I’m down with the idea that LLM’s have been especially successful because they ‘piggyback on language’ – our tool and protocol for structuring, compressing, and serialising thought, which means it has been possible to train LLM’s on compressed patterns of actual thought and have them make new language that sure looks like thought, without any direct experience of the concepts being manipulated, and if they do it well enough we will do the decompression, fleshing out the text with our experiential context. But I suspect that there are parts of my mind that also deal with concepts in an abstract way, far from any experiential context of the concept, just like the deeper layers of a neural network. I’m open to the idea, that just as the sparse matrix of an LLM is encoding connection between concepts without explicitly encoding edges, I think there will be multiple ways that we can look as the structure of an AI model and at our anatomy so that they are a squint and a transformation function away interesting overlaps. that will lead to and a kind of 'god of the gaps' scenario in which we conceptually carve out pieces of our minds as, 'oh the visual cortext is just an X', and deep questions about what we are.