[nonameiguess]

This seems to be misunderstanding what PAC learnable means. It's saying that for parameters epsilon and delta, you can get an algorithm for learning a hypothesis that, with probability 1 - delta, has an average error rate less than epsilon, given some number of training samples polynomial in 1/delta and 1/epsilon.

This has nothing to do with the mental model. The language model doesn't have to spit out probabilities. It can use a simple thresholding function and spit out certainties. They just might be wrong, but a human's "certain" mental models might be wrong, too.

[qayxc]

There's a very important difference, though: humans can and often do qualify their interpretation.

One major flaw of current PAC-based models is the inability to express certainty and being able to present the rationale behind their (lack of) confidence.

One crucial aspect of human language processing is the ability to question or interrogate oneself to adjust one's interpretation. During a dialog this can be done by actively asking for more context or confirmation.

So unlike current ML models, humans know that their interpretation/models might be wrong and can actively seek confirmation or correction.

A trivial example would be asking a digital assistant "What's the capital of Georgia?" There are two possible answers and there's no way of knowing what the "correct" one is without further context. So a human would likely either ask whether you mean the country of Georgia or the US state, or qualify their answer ("the capital of the country/the state capital is Tbilisi/Atlanta").

This is what GPT-J-6B returns:

  Q: What's the capital of Georgia?
  A: Atlanta

When given just the prompt "What's the capital of Georgia", GPT-J-6B also returns Atlanta and generates further text about the US state and its capital.

And it's not as if the model doesn't "know":

  Q: What is Tbilisi?
  A: Tbilisi is the capital of Georgia.

It's even more interesting when logic is involved:

  Q: If Albert is a pig and all pigs are blue, what colour is Albert?
  A: Albert is a blue pig.

I only asked for the colour, but close enough. Can GPT deal with ambiguity?

  Q: If Albert is a pig and all pigs are blue, is Albert happy?
  A: No.

Awesome! How about some common sense then:

  Q: If Alice is a dog and Bob just finished high school, can they have kids together?
  A: Yes, but they will have to get married first.

So yeah, GPT obviously has no actual language understanding. Bob having finished high school implies that he's a young human man while Alice is a dog. Humans and dogs can't have kids together.

But if the model doesn't understand language, why did it do so well on the logic test? In order to find out, I just asked :)

  Q: If Albert is a pig and all pigs are blue, why is Albert not happy?
  A: Because he is a pig.

Hm. So are all pigs unhappy then? Let's take them out of the equation and ask again:

  Q: If Albert is blue, is Albert unhappy?
  A: No, Albert is not unhappy.

Bingo! So the model only appeared to know the subtlety of "blue" having multiple meanings. It's as dumb as a rock and simply tried to match the tokens "is blue" against "is unhappy". Replacing "blue" with "pig" confirms this:

  Q: If Albert is a pig, is Albert unhappy?
  A: No, Albert is not unhappy. Albert is a pig.

Dad humour. Well, either that or just a dumb model capable of fooling us humans because we tend to interpret more intelligence into its replies than is actually there...

EDIT: in case you want to test this yourself (you might get different results, it's probabilistic after all):

https://6b.eleuther.ai

When using the Q/A-format, make sure not add a space after the "A:" prompt.

[moyix]

I find demos like yours hard to reason about because it is often the case that you can mess around with the prompt to get it to demonstrate understanding of things that it didn't seem to understand before. I wanted to see if it understood that humans and dogs can't have kids, and it does. Prompt is in [], generated text is in {}.

–––––––––––––––––––––––––––––––––

    [This is a question and answer series intended to demonstrate the logical reasoning capabilities of an AI system. The AI can understand ambiguity and will ask for more detail in such cases.

    Q: What's the capital of Georgia?
    A: That's ambiguous. Do you mean the country or the US state?

    Q: If Albert is a pig and all pigs are blue, what colour is Albert?
    A: Albert is a blue pig.

    Q: If Albert is a pig and all pigs are blue, is Albert happy?
    A: There is no logical connection between Albert's happiness and his color.

    Q: If Alice is a dog and Bob just finished high school, can they have kids together?
    A:]{ No.}

    [Q: Why can't they have kids together?
    A:]{ Because Alice is a dog and Bob just finished high school.}

    [Q: Why is that a problem?
    A:]{ Because Alice is a dog.

    Q: Why is that a problem?
    A: Because dogs can't have kids with humans.

    Q: Why can't dogs have kids with humans?
    A: Because dogs are animals and humans are humans.

    Q: Why are humans different from dogs?}

–––––––––––––––––––––––––––––––––

So does it understand after all? Or is trying different prompts a form of p-hacking here?

[qayxc]

> So does it understand after all? Or is trying different prompts a form of p-hacking here?

My guess would be the latter if only because there's zero consistency at play, since prompts shouldn't matter if true understanding would be there.

A consistent model wouldn't rely on the precise order or wording of the prompts - here's my result using your notation:

  [Q: Can humans and dogs have kids together?
  A:]{ This is an extremely touchy subject for some dog owners. The short answer is yes. The fact is that studies have shown that if a male and female dog are able to have a viable pregnancy, the offspring have a higher rate of survival. It is also important to note that the offspring are able to have puppies. This is important because it allows dogs to have kids of their own and increase the gene pool. }

Screenshot: https://ibb.co/Njg3BkC

So my money is on p-hacking. Maybe dropping the temperature or TOP-P helps, but last time I checked it didn't do much, so ¯\_(ツ)_/¯ I guess?

[qayxc]

Turns out dropping the temperature didn't help either, so yeah, p-hacking for the most part:

https://ibb.co/KNDyvsQ

[djjdj]

I disagree. Of course, it doesn't understand anything, it is a statistical language model. I also think that human jokes are mostly statistical in nature. That's why "Yes, but they will have to get married first" is funny. GPT was trained using texts from the Internet. It usually contains jokes and not logic predicates. At any prompt that has a structure of a joke, it will use (in the majority of the attempts) a pattern for a joke even if there is none. "If X is y, is X z?" is a typical joke pattern. If logical approach is applied in the future, it will still be derived from the statistics/reinforcement learning or something like that (the key word here is training/learning). The world is just too vast. If you want to have a resemblance of a logic model, you need a special dataset, but it will still just exploit patterns. It might be good enough, for practical use. It is arguable, if people mostly operate beyond patterns. Sorry if my text looks patronising or smth like that. I just assume.