[EForEndeavour]

The advanced capabilities of scaled up transformer models fed oodles of training data has burdened me with pseudo-philosophical questions about the nature of cognition that I am not well equipped to articulate, and make me wish I'd studied more neuroscience, philosophy, and comp sci earlier in life. A possibly off-topic thought dump:

- What is thinking, exactly?

- Does human (or superhuman) thinking require consciousness?

- What even is consciousness? Why is it that when you take a bunch of molecular physical laws and scale them up into a human brain, a signal pattern emerges that feels things like emotions, continuity between moments, desires, contemplation of itself and the surrounding universe, and so on?

- Why and how does a string predictor on steroids turn out to do things that seem so close to a practical definition of thinking? What are the best evidence-based arguments supporting and opposing the statement "GPT4 thinks"? How do people without OpenAI's level of model access try to answer this question?

(And yes, it's occurred to me that I could try asking GPT4 to help me make these questions more complete)

[Workaccount2]

> has burdened me with pseudo-philosophical questions about the nature of cognition that I am not well equipped to articulate, and make me wish I'd studied more neuroscience, philosophy, and comp sci earlier in life

Welcome to the club. There pretty much are no answers, just theories primarily played out as thought experiments. Its on of those areas where you can pick out who knows less (or is being disingenuous) by seeing who most confidently speaks about having answers.

We don't know what consciousness is, and we don't know what it means to "think". There, I saved you a decade of reading.

Edit: My choice theory is panpsychism, https://plato.stanford.edu/entries/panpsychism/ but again, we don't yet know how to verify any of this (or any other theory).

[ryanwaggoner]

It's interesting to me how many commenters on HN are absolutely convinced that GPT4 is incapable of thought or understanding or reasoning, it's "just" predicting the next word. And then they'll insist that it'll never be able to do things that it's already capable of doing...

Interestingly, more than one of these folks have turned out to be religious. I wonder if increasingly intelligent AI systems will be challenging for religious folks to accept, because it calls into question our place at the pinnacle of God's creation, or it casts doubt upon the existence of a soul, etc.

[nomel]

> because it calls into question our place at the pinnacle of God's creation, or it casts doubt upon the existence of a soul

I think this is a very simplistic view, that possibly suggests you haven't talked to many religious people.

I've never known a religious person who thought "thought" was the same as "soul", or that God is neccesarily a requirement for reasoning. Or that any of this is thought about much, considering it's so new.

Although, I suppose that if someone did say that God was a requirement for reasoning, a "logical within that context" perspective might be AI being some vicarious creation, since it wouldn't have been possible without us being able to reason.

I subscribe to the belief that reasoning is an eventual emergent law of nature/information. But, even that could, and does, fit into many "religious" perspectives perfectly well.

[dgb23]

If we could create a sentient being, it would be the first evidence of it being possible at all. If this casts doubt in the mind of a believer, then it tells us more about what belief is than anything else.

[anotherman554]

"Interestingly, more than one of these folks have turned out to be religious."

The guy fired by google for announcing LaMDA was sentient was religious.

I don't really see a meaningful distinction between declaring a machine is "thinking" for hand waving religious reasons and hand waving non-religious reasons, I'm afraid.

[defgeneric]

It's less unsettling when you think of LLMs as an approximation to a kind of "general intellect" recorded in language. But then the surprising thing is that we as "individual intellects" tend to operate the same way, perhaps more than we imagined.

[nicpottier]

The hypothesis that I find most compelling and intuitive is that language is thought and vice versa. We made a thing really good at language and it turns out that's also pretty good at thought.

One possible conclusion might be that the only thing keeping GPT algos from going full AGI is a loop and small context windows.

[robwwilliams]

Add the strange loops and embed in a body the interacts with a real or rich virtual word—that should do the trick. Of course there should ideally be an emotional-motivational context.

[narag]

- Does human (or superhuman) thinking require consciousness?

I was going to write this exactly. I believe these things think. They're just not alive.

- What even is consciousness?

My advice: stay as far as you can from that concept. Wittgenstein already noticed that many philosophical questions are nonsense and specifically mentioned how consciousness as felt from the inside is hopefully incompatible with any observation we make from the outside.

BS concepts like qualia are all the rage now, but ultimately useless.

[HarHarVeryFunny]

My views:

The best definition of "intelligence" is "the degree of ability to correctly predict future outcomes based on past experience".

Our cortex (part of the brain used for cognition/thinking) appears to be literally a prediction engine where predicted outcomes (what's going to happen next) are compared to sensory reality and updated on that basis (i.e. we learn by surprise - when we are wrong). This makes sense as an evolutionary pressure since ability to predict location of food sources, behavior of predators, etc, etc, is obviously a huge advantage over being directly reactive to sensory input in the way that simpler animals (e.g. insects) are.

I'd define consciousness as the subjective experience of having a cognitive architecture that has particular feedback paths/connections. The fact that there is an architectural basis to consciousness would seem to be proved by impairments such as "blindsight" where one is able to see, but not conscious of that ability! (eg. ability to navigate a cluttered corridoor, while subjectively blind).

It doesn't seem that consciousness is a requirement for intelligence ("ability to think"), although that predictive capability can presumably benefit from more information, so these feedback paths may well have evolutionary benefit.

The reason a "string predictor on steroids" turns out to be able to do things that seem like thinking is because prediction is the essence of thinking/intelligence! Of course there's a lot internally missing from GPT-4 compared to our brain, for example basics like working memory (any internal state that persists from one output word to the next) and looping/iteration, but feeding it's own output back in does provide somewhat of a substitute for working memory, and external scripting/looping (AutoGPT, etc) goes a long way too.

[kingkongjaffa]

I think since the mechanisms are different we should arrive at a distinction between:

organic thinking (I.e. the process our squishy human brains do)

and mechanical thinking ( the computational and stochastic processes that computers do ).

[TuringTest]

I don't think the substrate defines the nature of the thinking, but the form of the process does.

It is entirely possible to build mechanical thinking in organic material (think Turing machines built on growing tissue), and it could also be possible to build complex self-referential processes simulated on electronic hardware, of the kind high-level brains do, with their rhythms of alfa and beta waves.

[coding123]

> What even is consciousness? Why is it that when you take a bunch of molecular physical laws and scale them up into a human brain, a signal pattern emerges that feels things like emotions, continuity between moments, desires, contemplation of itself and the surrounding universe, and so on?

I doubt we'll ever be able to answer this, even after we create AGI.