[siglesias]

Your vocabulary presupposes the categories you’re asserting are equivalent. The process of evolution and AI training are vastly different. One confers a survival advantage and is suffused with values that are essential to humans, such as morality, the primacy of vision, taste and smell, etc. AI training is an attempt to transfer functions that allow for human survival and flourishing to objects that are not human. AI training, and especially the Turing Test featured in the Chinese room is about mimicking humans and human evolution is about survival and forms the basis of our aesthetic and moral judgments. One is simply a simulation of the other. Consciousness might not matter to what you concern yourself with as somebody amazed with AI (I am as well), but surely you believe that there is a moral difference between harming a human and harming an LLM, even verbally. What do you think accounts for that, if not consciousness?

[Kim_Bruning]

> but surely you believe that there is a moral difference between harming a human and harming an LLM, even verbally.

I'm becoming less sure of this over time. As AI becomes more capable, it might start being more comparable to smaller mammals or birds, and then larger ones. It's not a boolean function, but rather a sliding scale.

Despite starting out from very skeptical roots, over time Ethology has found empirical evidence for some form of intelligence in more and more different species.

I do think that this should also inform our ethics somewhat.

[siglesias]

As I've argued elsewhere, we should care what the source of the behavior is. The reason expand ethical concern to dogs and birds even though they don't have the capability to use language and why we don't to LLMs, even though they use language very ably, is precisely because we recognize the biological causes of consciousness. The reason we keep getting confused about whether these concerns apply to AI is because we apply a behavioral standard rather than the standard we use everywhere else, which is a biological one. We have higher certainty that dogs are conscious, yes, because of their behavior, but also, and critically, because they share biology with us.

[Kim_Bruning]

If you're going to refer to biology, be aware that the relevant subfield that defines the biological standard is in fact called Ethology. To attain rigor, Ethology historically rejected anthropomorphism in favor of strict behavioral evidence, seeing as that is the primary empirically measurable evidence available.

On a side note: it's been a pleasure reading through the debates with you, and possibly we can continue over mail!