[achow]

It is perhaps very relevant what Chris Olah (Anthropic co-founder) said to pope..

I am a scientist. I lead a research team that studies the internal structure of these models—what is actually happening inside them. And I will be honest: we keep finding things that are mysterious, even unsettling. We find structures that mirror results from human neuroscience. We find evidence of introspection. We find internal states that functionally mirror joy, satisfaction, fear, grief, and unease.

https://www.anthropic.com/news/chris-olah-pope-leo-encyclica...

[boppo1]

Yeah that reeks of marketing to me. This guy may believe all those things, but his getting paid may also depend upon his believing them.

[ch4s3]

This is an incredibly strong claim and I’ve never seen any particularly strong evidence for them beyond “trust me, I’ve seen into the abyss.”

[mrdomino-]

I think what you've said here is unfair and overly cynical; nowhere have I read Chris or any of the Anthropic people make the claim that LLMs definitively are conscious. What they say is that there is increasing uncertainty, and evidence - for which they show receipts - backing up that uncertainty.

Personally, I have been unconvinced by any of these definitive yes or no answers to "are LLMs conscious?" - in both directions. The "they are not / can not be conscious" side relies too heavily on mechanistic reductionist arguments that can apply equally to neurochemical processes in the human brain - and yet, it seems that humans, with brains made of these neurochemical processes, are conscious. At the same time, the "they are definitely conscious" answers seem to generally rely too heavily on self-deception and lack of reality testing.

To be able to say something definitive here, it seems to me that one would need to say definitively what this experience is. I have not heard any of the loud voices in the arena - not Chiang, not Giulio Tonini, not Karl Friston, etc - do so. Therefore I find Anthropic's uncertainty, and careful, caring investigative process, well grounded given the evidence.

[ch4s3]

My aim isn't cynicism so much as an interest in clear an evidence based information about these systems which are clearly important technological developments for the future. It may turn out that there is something analogous to cognition going on and that would be quite significant and require us to think differently about the technology, but we need to be very sure of something like that.

> Therefore I find Anthropic's uncertainty, and careful, caring investigative process, well grounded given the evidence.

I'm inclined to agree mostly, but from the outside its impossible to separate marketing, belief, scientific inquiry, and just plain enthusiasm. Its all to opaque.

[mrdomino-]

I will concur that it's too opaque.

[irishcoffee]

The point here, Chris Olah is literally claiming he has seen into the abyss. Who are you disagreeing with?

[ch4s3]

I'm saying that the burden of proof is on Chris Olah.

[astockwell]

This is the fundamental trust challenge with private/profitability-driven 'science'.

[mrdomino-]

And it is worth noting, I think, that Olah's message to the pope is centrally making that point: that Olah and his peers are within incentive structures that distort their actions and understanding, and therefore that dialog with those outside of such incentive structures is necessary (not sufficient) for their outputs to be trustworthy.

[ikrenji]

well of course anything from anthropic on the topic is going to be bombastic and hyped to the moon

[ezst]

I mean, this reminds me of the early imagenet days, when people were first trying to explain the unreasonable efficiency of neural networks at interpreting content in images. They found out that, after enough backprop, some layers of the network became specialized in contours extraction, shape extraction, etc, in ways that can be seen as analogous to earlier CV techniques (Canny/Hough transforms, ...).

Later, Google was having fun feeding whole bunch of youtube content to artificial neural networks, unsupervised, and figured that certain parts of the network would, too, specialize, only to have the activation functions be run backwards and render an abstract image of a cat¹.

None of that is terribly new or surprising for anyone having studied and dealt with neural networks. The only difference today is that the field has completely flip flopped from approaching the subject with scientific rigor and cautious excitement to being a clueless billionaire infinite money printing machine fed on deceiving anthropomorphism and FUD.

¹: https://blog.google/innovation-and-ai/products/using-large-s...

[insanitybit]

Total nonsense.