[Workaccount2]

> has burdened me with pseudo-philosophical questions about the nature of cognition that I am not well equipped to articulate, and make me wish I'd studied more neuroscience, philosophy, and comp sci earlier in life

Welcome to the club. There pretty much are no answers, just theories primarily played out as thought experiments. Its on of those areas where you can pick out who knows less (or is being disingenuous) by seeing who most confidently speaks about having answers.

We don't know what consciousness is, and we don't know what it means to "think". There, I saved you a decade of reading.

Edit: My choice theory is panpsychism, https://plato.stanford.edu/entries/panpsychism/ but again, we don't yet know how to verify any of this (or any other theory).

[ryanwaggoner]

It's interesting to me how many commenters on HN are absolutely convinced that GPT4 is incapable of thought or understanding or reasoning, it's "just" predicting the next word. And then they'll insist that it'll never be able to do things that it's already capable of doing...

Interestingly, more than one of these folks have turned out to be religious. I wonder if increasingly intelligent AI systems will be challenging for religious folks to accept, because it calls into question our place at the pinnacle of God's creation, or it casts doubt upon the existence of a soul, etc.

[nomel]

> because it calls into question our place at the pinnacle of God's creation, or it casts doubt upon the existence of a soul

I think this is a very simplistic view, that possibly suggests you haven't talked to many religious people.

I've never known a religious person who thought "thought" was the same as "soul", or that God is neccesarily a requirement for reasoning. Or that any of this is thought about much, considering it's so new.

Although, I suppose that if someone did say that God was a requirement for reasoning, a "logical within that context" perspective might be AI being some vicarious creation, since it wouldn't have been possible without us being able to reason.

I subscribe to the belief that reasoning is an eventual emergent law of nature/information. But, even that could, and does, fit into many "religious" perspectives perfectly well.

[dgb23]

If we could create a sentient being, it would be the first evidence of it being possible at all. If this casts doubt in the mind of a believer, then it tells us more about what belief is than anything else.

[anotherman554]

"Interestingly, more than one of these folks have turned out to be religious."

The guy fired by google for announcing LaMDA was sentient was religious.

I don't really see a meaningful distinction between declaring a machine is "thinking" for hand waving religious reasons and hand waving non-religious reasons, I'm afraid.

[defgeneric]

It's less unsettling when you think of LLMs as an approximation to a kind of "general intellect" recorded in language. But then the surprising thing is that we as "individual intellects" tend to operate the same way, perhaps more than we imagined.