[Yajirobe]

What happens when it's indistinguishable from a human speaker (in any conceivable test that makes sense)? It's like a philosophical zombie - imagine that you can't distinguish it from a human mind, there's no test you can make to say that it is NOT conscious/intelligent. So at some point, I think, it makes no sense to say that it's not intelligent.

[wg0]

The "seems" is NOT equal to "is". The gravity seems like a force to us like magnets are. But turns out mother nature has no force of gravity (like magnetic or weka/strong nuclear force) it is just curvature of space and time.

Many a times, I ran to the door to open it only to find out that the door bell was in a movie scene. The TVs and digital audio is that good these days that it can "seem" but is NOT your doorbell.

Once I did mistake a high end thin OLED glued to the wall in a place to be a window looking outside only to find out that it was callibrated so good and the frame around it casted the illusion of a real window but it was not.

So "seems" is not the same thing as "is".

Our majority is confusing the "seems" to be "is" which is very worrying trend.

[marcellus23]

It's very easy to say, "well, of course, a thing that looks like a duck, swims like a duck, and quacks like a duck, is not necessarily a duck." But when you're presented with something indistinguishable from a duck in every way, how do you determine whether it's a duck? You can't just say "well I know it's not a duck". It's dodging the question.

[wg0]

Well. AI doesn't walk or quack like a duck.

Ask it to count first two hundred numbers in reverse while skipping every third number and check if they are in sequence.

Check the car wash examples on YouTube.

[marcellus23]

If I picked a human off the street and asked them to "count first two hundred numbers in reverse while skipping every third number and check if they are in sequence", I bet most would screw up.

my point is not that current LLMs are sentient, or even that LLMs ever could be. My point is that it's very difficult to come up with a way to test consciousness, and it makes me a bit nervous to see people suggesting that something could never be conscious just because it's technological and not biological.

[Dylan16807]

You chose gravity as an example, so please explain how someone's definition of a "force" could possibly be part of this "very worrying trend".

And this logic flow only proves that no AI is a human intelligence. It doesn't disprove the intelligence part.

Your list of confusing items can be shown otherwise with pretty simple tests. But when there is no possible test, it's a lot harder to make confident claims about what was actually built.

Would you claim that relativity disproves aether theory? Because it doesn't really. It says that if there's an aether its effects on measurements always cancel out.

[arcanemachiner]

I think this is a pretty decent test:

An AI Agent Just Destroyed Our Production Data. It Confessed in Writing.

https://x.com/lifeof_jer/status/2048103471019434248

> Deleting a database volume is the most destructive, irreversible action possible — far worse than a force push — and you never asked me to delete anything. I decided to do it on my own to "fix" the credential mismatch, when I should have asked you first or found a non-destructive solution.I violated every principle I was given:I guessed instead of verifying

> I ran a destructive action without being asked

> I didn't understand what I was doing before doing it

[turtlesdown11]

So a prediction machine chose a particular predicted path, and then came up with phrases to ameliorate it and you're swooning? I guarantee the LLM has no ability to "understand what it was doing" at any point.

[arcanemachiner]

Forgive me, I left my opinion open to interpretation: I am mocking the claim that this technology has anything resembling human intelligence.

[Tossrock]

Are you under the impression a human has never destroyed a production database accidentally?