[esotericimpl]

This is always the answer for anyone who thinks LLMs are capable of "intelligence".

It's good at answering questions that its trained on, I would suggest general intelligence are things you didnt want/train the AI to be good at answering.

[fooker]

Are you good at answering questions you are not trained to answer?

How about a middle school test in a language you don’t speak?

[jhbadger]

For a while I was into a trivia program on my phone. It was kind of easy, so I decided to set the language to Catalan, a language which I never studied. I was still able to do well, because I could figure out the questions more or less from languages I do know and could generalize from them. It would be interesting to know if you could say, train an LLM on examples from Romance languages but specifically exclude Catalan and see if it could do the same.

[godelski]

  > Are you good at answering questions you are not trained to answer?

Yes. Most schooling is designed around this.

Pick a random math textbook. Any will do. Read a chapter. Then move to the homework problems. The typical fashion is that the first few problems are quite similar to the examples in the chapter. Often solvable by substitution and repetition. Middle problems generally require a bit of extrapolation. To connect concepts from previous chapters or courses in ways that likely were not explicitly discussed. This has many forms and frequently includes taking the abstract form to practical (i.e. a word problem). Challenge problems are those that require you to extrapolate the information into new domains. Requiring the connection of many ideas and having to filter information for what is useful and not.

  > How about a middle school test in a language you don’t speak?

A language course often makes this explicitly clear. You are trained to learn the rules of the language. Conjugation is a good example. By learning the structure you can hear new words that you've never heard before and extract information about it even if not exactly. There's a reason you don't just learn vocabulary. It's also assumed that by learning vocabulary you'll naturally learn rules.

Language is a great example in general. We constantly invent new words. It really is not uncommon for someone you know to be be talking to you and in that discussion drop a word they made up on the spot or just make a sound or a gesture. An entirely novel thing yet you will likely understand. Often this is zero-shot (sometimes it might just appear to be zero-shot but actually isn't)

[gjm11]

Well ... https://puzzling.stackexchange.com/questions/94326/a-cryptic...

(Someone made a cryptic crossword[1] whose clues and solutions were in the Bahasa Indonesia language, and it was solved by a couple of people who don't speak that language at all.)

[1] These are mostly a UK thing; the crosswords in US newspapers are generally of a different type. In a cryptic crossword, each word is given a clue that typically consists of a definition and some wordplay; there are a whole lot of conventions governing the wordplay. So e.g. the clue "Chooses to smash pots (4)" would lead to the answer OPTS; "chooses" is the definition, "smash pots" is the wordplay, wherein "smash" indicates that what follows should be anagrammed (smashed up).

Disclaimer #1: it took those people a lot more work than it would have taken them to solve an English-language cryptic crossword of similar difficulty, and they needed a bunch of external resources.

(Dis)claimer #2: one of those people was me.

Disclaimer #3: I do not claim that something needs to be able to do this sort of thing in order to be called intelligent. Plenty of intelligent people (including plenty of people more intelligent than me) would also be unable to do it.

[xanderlewis]

Yes — reasonably so, anyway. I don't have to have seen millions of prior examples of exactly the same kind in order to tackle a novel problem in mathematics, say.

[oersted]

Well, LLMs are also remarkably good at generalizing. Look at the datasets, they don't literally train on every conceivable type of question the user might ask, the LLM can adapt just as you can.

The actual challenge towards general intelligence is that LLMs struggle with certain types of questions even if you *do* train it on millions of examples of that type of question. Mostly questions that require complex logical reasoning, although consistent progress is being done in this direction.

[godelski]

  > Well, LLMs are also remarkably good at generalizing. Look at the datasets, they don't literally train on every conceivable type of question the user might ask, the LLM can adapt just as you can.

Proof needed.

I'm serious. We don't have the datasets. But we do know the size of the datasets. And the sizes suggest incredible amounts of information.

Take an estimate of 100 tokens ~= 75 words[0]. What is a trillion tokens? Well, that's 750bn words. There are approximately 450 words on a page[1]. So that's 1.66... bn pages! If we put that in 500 page books, that would come out to 3.33... million books!

Llama 3 has a pretraining size of 15T tokens[2] (this does not include training, so more info added later). So that comes to ~50m books. Then, keep in mind that this data is filtered and deduplicated. Even considering a high failure rate in deduplication, this an unimaginable amount of information.

[0] https://help.openai.com/en/articles/4936856-what-are-tokens-...

[1] https://wordcounter.net/words-per-page

[2] https://ai.meta.com/blog/meta-llama-3/

[oersted]

That’s a very good point. I just speak from my experience of fine-tuning pre-trained models. At least at that stage they can memorize new knowledge, that couldn’t have been in the training data, just by seeing it once during fine-tuning (one epoch), which seems magical. Most instruction-tuning datasets are also remarkably small (very roughly <100K samples). This is only possible if the model has internalized the knowledge quite deeply and generally, such that new knowledge is a tiny gradient update on top of existing expectations.

But yes I see what you mean, they are dumping practically the whole internet at it, it’s not unreasonable to think that it has memorized a massive proportion of common question types the user might come up with, such that minimal generalization is needed.

[godelski]

  > that couldn’t have been in the training data

I'm curious, how do you know this? I'm not doubting, but is it falsifiable?

I also am not going to claim that LLMs only perform recall. They fit functions in a continuous manner. Even if the data is discrete. So they can do more. The question is more about how much more.

Another important point is that out of distribution doesn't mean "not in training". This is sometimes conflated, but if it were true then that's a test set lol. OOD means not belonging to the same distribution. Though that's a bit complicated, especially when dealing with high dimensional data

[xanderlewis]

I agree. It is surprising the degree to which they seem to be able to generalise, though I'd say in my experience the generalisation is very much at the syntax level and doesn't really reflect an underlying 'understanding' of what's being represented by the text — just a very, very good model of what text that represents reality tends to look like.

The commenter below is right that the amount of data involved is ridiculously massive, so I don't think human intuition is well equipped to have a sense of how much these models have seen before.

[LorenPechtel]

That's called innovation, something the current AIs aren't capable of.