[tasuki]

This is possibly the least insightful article I have read on HN. My comment is just a rant against the many misguided points it attempts to make...

> Welfare is defined as "the health, happiness, and fortunes of a person or group".

What about animals? Isn't their welfare worthy of consideration?

> Saying that there is no scientific consensus on the consciousness of current or future AI systems is a stretch. In fact, there is nothing that qualifies as scientific evidence.

There's no scientific evidence for the author of the article being conscious.

> The issue is, if we push moral considerations for algorithms, we will not end up with a higher regard to human welfare.

Same with animals. Doesn't mean it's not worthwhile.

> However, the implication is clear: LLMs are basically the same as humans.

No: there's no such implication.

> Already now, it is a common idea among the tech elite is that humans as just a bunch of calculations, just an LLM running on "wetware". It is clear that this undermines the belief that every person has inalienable dignity.

It is not clear to me how this affects inalienable (?) dignity. If we aren't just a bunch of calculations, then what are we?

> And if a human being is not much more than an algorithm running on meat, one that can be jailbroken and exploited, then it follows that humans themselves will increasingly be treated like the AI algorithms they create: systems to be nudged, optimized for efficiency, or debugged for non-compliance. Our inner lives, thoughts, and emotions risk being devalued as mere outputs of our "biological programming," easily manipulated or dismissed if they don't align with some external goal. Nobody will say that out loud, but this is already happening

Everyone knows this is already happening. It is not a secret, nor is anyone trying to keep it a secret. I agree it is unfortunate - what can we do about it?

> I've been working in AI and machine learning for a while now.

Honestly, I'm surprised. Well done.

[indigo945]

Just to add on to this, because I agree with all points you make: the article argues that "people chatting with Claude a lot and asking if it feels conscious [...] is neither good AI research, nor a deep philosophical investigation", but then builds half its argument on a second-rate sci-fi novel:

    > The sci-fi novel Permutation City captures the absurd endpoint of 
    > this logic when a simulated mind considers its own nature: "And if 
    > the computations behind all this had been performed over millennia, 
    > by people flicking abacus beads, would he have felt exactly the same? 
    > It was outrageous to admit it—but the answer had to be yes." [...] 
    > A theory that demands we accept consciousness emerging from millennia of 
    > flickering abacus beads is not a serious basis for moral consideration; 
    > it's a philosophical fantasy.

Concluding from a random novel the author has read that an argument that actual philosophers working in actual academia on actual problems of Philosophy of Mind is invalid because, to wit, it feels invalid -- I assume that's the "good AI research" and "deep philosophical investigation" the author was looking for, then?