[throwaway713]

Suppose one selects an arbitrary hot-button issue [X] with two opposing sides and one side has anything less than overwhelming support. And then that person writes an article titled "Side 1 of issue [X] is true". Not "maybe" or "possibly". Just a straight-up declaration by fiat.

Would you categorize this particular style of rhetoric to be persuasive or annoying? And before you say "persuasive" because you're thinking about this specific issue regarding AI consciousness, consider many things in the past that have been written as though they were absolutely definitive, and yet today we believe exactly the opposite, and for many such issues we find the prevailing viewpoint at the time reprehensible.

That's not to say that Ted is wrong at all here; I'm not commenting on that. But I find the entire style of the article grating because it seems to violate common assumptions regarding "good faith" debate, and I would find the article equally frustrating if he had titled it "Artificial intelligence is conscious" and argued the opposite side, albeit in the same tone and using the same persuasion devices.

[swatcoder]

> Would you categorize this particular style of rhetoric to be persuasive or annoying?

Why are those the choices?

Essays are situated along countless dimensions: tone, vocabulary, author, zeitgeist, publication context, intent, subtext, relationship to other works and expressions, etc

A "good faith" reader takes all of those into consideration as they absorb the essay, and integrate it with their own intellectual situation that sits along just as many countless dimensions.

Nobody's asked to sign a notarized binding document that they wholesale agree or disagree with everything said in this essay -- or its conclusions. Nor are they obliged to have some strong reaction to it at all, let alone annoyance.

It's just one among thousands of essays about a "hot button topic", to be taken however it's personally received.

Why should Chiang have to take responsibility for making sure it's not too strongly positioned for your persomal taste. Maybe he really does see it so clearly and is simply being earnest. Maybe he enjoys the literary flourish of prose in strong language. Maybe he just wants to express something as a prose-poetic human, not maximize persuasion or non-annoyance per se.

[eberkund]

It's only declaration by fiat if you stop reading at the end of the title. Should authors add a qualifier like in my opinion to every statement that they make?

The essay structure you're criticizing is exactly how I was taught to write from primary school through to university. You start with a title or hook, introduce the topic and propose a thesis. That is followed up upon with supporting arguments for the primary claim.

[nyeah]

This is a reasonable reaction to the title, but not to TFA.

[runarberg]

I don‘t see the problem here. Newton could have just as well published an article titled: “Objects with mass attract each other” and Darwin famously wrote a whole book titled “On the Origin of the Species by Means of Natural Selection” which is just another way of saying: “Natural Selection is How Species Evolve”.

[enduser]

He famously wrote a book with that title, but he actually wrote a book titled On the Origin of Species by Means of Natural Selection.

[runarberg]

Thats what I meant. I even looked it up before posting. I suspect that definite article must have slipped in because I‘m used to the Icelandic translation which uses a definite article in the title.

[Barrin92]

>and one side has anything less than overwhelming support

except that's not the case here. Chiang is explaining and reiterating what is the position that has overwhelming support on the question, and the people he is arguing the opposite side sound like this, which he helpfully quoted in the article

"Amanda Askell (who is credited as a lead author of Claude’s constitution), said, “I want Claude to be very happy—and this is a thing that I want Claude to know more, because I worry about Claude getting anxious when people are mean to it on the internet and stuff"

When the person you're arguing with sounds like an eight year old girl talking about her toy teddy I think Ted Chiang is if anything being charitable, if you're of a more honest and straight-forward persuasion you might argue these people belong into a mental health clinic not in charge of technological infrastructure

[handoflixue]

> When the person you're arguing with sounds like an eight year old girl

Wow, what a cheap ad-hominem. Do you have an actual point?

[Barrin92]

>Do you have an actual point?

Yes, the same one that's made in the article. Anxiety and happiness are emotional, sensory, somatic states as a consequence of evolved and embodied traits and biochemistry in animals. Saying Claude is anxious or happy is like saying my TI-83 is mad if it can't solve an equation or my thermometer is in pain if it touches a hot stove.

I wasn't making an ad hominem attack, her thinking is quite literally that of a child who sees a system output 'sad text' and, like someone seeing a sad expression on a stuffed teddy, concludes that this is a property of the object rather than her own emotional reaction.

[adjejmxbdjdn]

And how did that evolve and how does it work?

Given that we don’t know how consciousness works how have you concluded that it’s certainly not an emergent property of something like a highly trained LLM?

[Barrin92]

>how have you concluded that it’s certainly not an emergent property of something like a highly trained LLM?

the same way I (and likely you) have concluded it for anything else. We don't assume objects that share no similarity with human or animal physiology or evolutionary development are conscious (let alone happy or anxious). 'Emergence' isn't a word you can abuse to justify your a priori assumptions in the absence of an explanation or even reason to assume something exists.

We can say a chemical property emerges from the configuration of a molecule because we can explain the process by which it does and observe the property, when people claim that consciousness "emerges" from an LLM they posit that it is conscious, and use "emergence" as a gap-filler to explain away the need for the process by which that allegedly occurs.

If you want to know the neurophysiology or evolutionary biology of pain or anxiety, which we do know quite well you can find them in a textbook, but suffice to say transformer models don't share any of them.

And importantly if anyone seriously believed transformer models were capable of conscious experience as Chiang points out they would have been in despair when AlphaFold was released, it's structurally a virtually identical system. But nobody did, because it didn't 'talk to them' through a chat interface.

[hgfa-xx]

Yes, the point is that Askell's argument is like that of an 8 year old girl.

[cataphract]

You must hate reading legal briefs.

[pixelpoet]

100% with you, it degenerates to proof by authority if someone popular / with clout just gets to declare "nuh uh".

I furthermore think it's ridiculous for humans to declare that our brains have a monopoly on certain patterns of electrical signals (if we reject supernaturalism).

> The result is a sentence-continuation machine that is likelier to emit sentences resembling those that a thoughtful, moral person could utter.

And we're 100% certain that humans aren't just as equally reduced to "stochastic parrots", if we're going to be infinitely reductive?

I don't believe that current AIs are conscious, but I think it's incredibly naive to take a strong stance on any future AI; it's much like the difference between atheism and agnosticism.

[runarberg]

You are referring to an Onion article in the lead up the Iraq War This War Will Destabilize The Entire Mideast Region And Set Off A Global Shockwave Of Anti-Americanism vs. No It Won’t[1] and you are painting Ted Chiang’s point in This Fine Article as Bob Sheffer’s counterpoint in the Onion’s piece.

However, I see a problem with that comparison. The debate here is on a philosophical matter in field in which Chiang is an extremely influential figure and his opinion are taken seriously. Second Chiang’s reasoning is extremely well argued, defining each term, explaining each nuance, citing other experts, etc. And finally, and most importantly, in The Fine Article, and unlike Bob Sheffer in the Onion Piece, Chiang entertains the possibility that he is wrong and his critics are right, explores the implications and reaches conclusions based on them:

> Being open to the possibility that LLMs are conscious is the same as being open to the possibility that Microsoft Word is conscious, or, more precisely, that multiple distinct consciousnesses are dormant in every Word document containing a conversational transcript, and that they are awakened every time the document is loaded.

I think you are wrong in painting Chiang’s argument as a belief in human exceptionalism. The thing to know about our brains (and the brains of other animals) is that they are not digital computers, and they are not even statistical inference machines. And as such they can be extremely optimized in doing the computations (or any state manipulations) required for the quality of life of the individual and the species as a whole (and their companion species).

1: https://theonion.com/this-war-will-destabilize-the-entire-mi...

[shafoshaf]

The problem I have with good faith debate is that it often falls into a fallacy of "fair-time" meaning we think we have to give the other side equal time. This because obvious with things like the Holocaust. Or when you have a legal person (e.g. RFK Jr.) asking to debate a scientist (e.g. Dr. Peter Hotez.)

[throwaway713]

> The problem I have with good faith debate is that it often falls into a fallacy of "fair-time" meaning we think we have to give the other side equal time

The Catholic church could have said exactly the same thing at one point. "Why should we even devote time to an argument as absurd as the earth not being the center of the universe?" There are darker examples along the lines of those you give, with beliefs quite opposite to those we have nowadays.

[runarberg]

Galileo’s (or rather the Copernican) model was still wrong though. It had obvious flaws and the church was not wrong in keeping their older model of the universe while a better model (Kepler‘s model) was still in the works.

What Galileo was asking the church to do was extremely unreasonable. He was basically asking them to throw a way a model which had worked fine for hundreds of years just because he observed the phases of Venus and moons of Jupiter. I mean would you? Especially for a model which was worse at predicting the motions of the planets.

Had Galileo’s model been better then Ptolemies’ I could see a case for his arguments, but it wasn’t, and there was no reason for the church to take his arguments at equal value with those in favor of keeping the Ptolemaic model.