Thought is a derivative of sensory processing. LLM does not have a physical body to interact with the world, nor does it develop itself and learn anything by experiencing the world, it has no subjective experience or subjective feeling, it has no qualia, it's symbols are not grounded in physical reality and it's "thoughts" is a mere simulacrum. Anyone personifying an LLM is just derealised by convincing outputs, not realising that manipulating symbols according to rules does not imply understanding
I mean, there are still philosophers metaphorically fist fighting about this stuff. Last time I stepped into the fray on this topic I got clapped back by someone from an area of philosophy of mind from after I graduated. It was an interesting perspective that was unaware of, but I studied language, not mind:
> you randomly sample letters from the alphabet and those letters make up actual words, then actual sentences
That sounds like a decently apt description of how I (a human) communicate. The only thing is that I suppose you implied a uniform distribution, while my sampling approach is significantly more complicated and path-dependent.
But yes, to the extent that I have some introspective visibility into my cognitive processes, it does seem like I'm asking myself "which of the possible next letters/words I could choose would be appropriate grammatically, fit with my previous words, and help advance my goals" and then I sample from these with some non-zero temperature, to avoid being too boring/predictable.
"it" is also not "thinking". It is still randomly (though not all words are equal probabilities) sampling from a distribution of words that have been stolen and it been trained on
If "randomly sampling from a trained distribution" can't produce useful, meaningful output, then deterministic computation is even more suspect. After all, it's a strict subset. You're sampling with temperature zero from a handcrafted distribution.
(this post directionality ok, but there's many a devil in the details)