[mannykannot]

We cannot assume that, because text generation is all these models do, then it must be possible to get answers to the questions we want to ask by examining their textual responses.

It is fair to ask why, if we accept these verbal challenges as good evidence for a theory of mind in children, we would not accept them for these models, but children have nothing like the memory for text that these models have, and the corpus of text that these models have been trained on includes a great many statements that tacitly represent their authors' theory of mind (i.e. they are the sort of statements that would typically be made by someone having a theory of mind, just as arithmetically-correct statements concerning quantities are to be expected from people who know arithmetic.)

To be clear, I am not arguing that it would be impossible to show a theory of mind in a system that can only interact through text, but personally, I think it will require a model with greater capabilities than responding to prompts. For example, when models can converse among themselves, I think we will know.

[rhn_mk1]

> To be clear, I am not arguing that it would be impossible to show a theory of mind in a system that can only interact through text

I think you are, because

> a model with greater capabilities than responding to prompts

interacts in other ways than text.

Even then, I don't see what's so special about language that it needs to be separated from other ways of interaction. If language is not enough to derive empirical answers, why should physical movements or radio emissions be?

Even if you don't assume that it's necessarily impossible to get the answers empirically for a text-based model, you must keep in mind that that option is open. Perhaps we will never find out if language models have a theory of mind.

However, judging by the discussions around the topic, very few people highligh the unknowability. If I have to choose between "yes" or "no" while the reality is "maybe", I'd choose a "yes" purely out of caution.

[mannykannot]

Two models having a coherent conversation - a scenario which follows directly from my post - would be a purely textual example of what I mean.

> Perhaps we will never find out if language models have a theory of mind.

We appear to be in agreement here.

When the state of our knowledge is 'maybe', it seems rash to assume either 'yes' or 'no'.

[rhn_mk1]

What does it change when you add another model? I don't see how this lets us extract extra information.

What distinguishes two conjoined models from one model with a narrowing across the middle?

If the idea is to have two similar minds building a theory of each other, then I guess this could be informative, but first we'd have to establish that the models are "minds" in the first place. It's not clear to me what that requires.

[mannykannot]

Here's where I am coming from: there have been a number of experiments to teach language to other species, but there is always a problem in trying to figure out to what extent they 'get' language - For example, there is the case of the chimpanzee Washoe signing "water" and "bird" on first seeing a swan - was it, as some people contended, inventing a new phrase for picking out swans (or even aquatic birds in general), or was it merely making the signs for two different things in the scene before it? [1]

One thing that has not been seen (as far as I know) is two or more of these animal subjects routinely having meaningful conversations among themselves. This would be a much richer source of data, and I do not think it would leave much doubt that they 'got' language to a very significant degree.

[1] https://www.nybooks.com/articles/2011/11/24/can-chimps-conve...