[lucubratory]

I'm not sure if you're contradicting yourself or if I just haven't understood what you've written, so I'd like to ask two clarifying questions. The first is: Are you claiming that any system built on this architecture (i.e. an LLM is a crucial part of what makes it behave intelligently) is definitionally not thinking, and this would be the case regardless of any evidence to the contrary?

If that isn't what you are claiming, what evidence would convince you that an AI system which critically depended on LLMs to function is thinking?

[ggm]

I'd need a theory as to why intelligence is in an llm not just apparent stimulus response outcomes. There is a huge corpus of knowledge inside the model and a black box for most of us. It's generating syntactically correct phrases, reams of them but with significant errors which to me say there is no apparent understanding of the regurgitated facts, and I see no introspection or intent.

Evidence is a more abstract concept. Nothing to date is evidence, to me. I guess it will be settled in hindsight and a theory to me is the huge missing link. We do (as far as I know) have a distinct lack of theories about intelligence at large, I find it encouraging how much recent research sees long term evidence of intelligence in higher organisms and better understanding for brain function but without being either a brain scientist or an fMRI expert I think I can say neither brain science nor fMRI science think they've cracked it yet, and so llm intelligence doesn't even begin to have theories grounded in organic, existent intelligence to work on just observations of input and output.

[lucubratory]

You might be interested in global workspace theory as a theory of consciousness. You can find more discussion of it, and other theories of consciousness as they relate to modern AI, in this very helpful paper: https://arxiv.org/abs/2308.08708