[lovemenot]

This argument is imbalanced. Not necessarily wrong.

To follow scientific method, you should specify what would be meaningful according to your parameters: not merely parroting. If and when that is achieved by LLMs, instead of moving the goalposts, accept that something genuine has happened.

>> We will be flooded with content that will get harder and harder to prove wrong, but it will be wrong.

I mostly agree, especially about quantity. But since you wish to draw a line in the sand, define quality. Or truth.

[jarek83]

>> But since you wish to draw a line in the sand, define quality. Or truth.

I asked it about tourists attractions in a city I know. One of the items it listed was a Zoo. There is no Zoo there. It seemed like it just took the majority of the question into account at some point and ignored the city name itself and since most questions like that are about big cities that have zoos, it went ahead to suggest it, since it was among probable answers.

It will get better with more input and will get this kind of obvious questions right. But I believe it will never truly 'know' what it is talking about. And we will start to believe it does.

[fl7305]

Exactly.

Or to put it differently: Is "GPT-4 is just a parrot" a falsifiable claim?

If so, what does the test look like that would falsify it?

[jarek83]

I guess debugability will be its most desired feature soon. Let's see what copyrights will do with it.

[fl7305]

What do "debuggability" or "copyrights" have to do with:

Is "GPT-4 is just a parrot" a falsifiable claim?

If so, what does the test look like that would falsify it?

Can you think of questions to test GPT-4s ability to reason that 90% of human 12-year olds will answer correctly, but where GPT-4 constantly struggles?

[jarek83]

Look at my previous comment about a city without a Zoo.