[skydhash]

They're not producing anything new... If you give it the answer before asking the question, no wonder it can answer. Prompting is to find resonance in the patterns extracted from the training data, which is why it fails spectacularly for exotic programming languages.

[vidarh]

When you invent a language and tell it express something in that language, you've not given it the answer before asking the question.

That's an utterly bizarre notion. The answer in question never existed before.

By your definition humans never produce anything new either, because we always also extrapolate on patterns from our previous knowledge.

> it fails spectacularly for exotic programming languages.

My experience is that it not just succeeds for "exotic" languages, but for languages that didn't exist prior to the prompt.

In other words, they can code at least simple programs even with zero-shot by explaining semantics of a language without giving them even a single example of programs in that language.

Did you even read the comment you replied to above?

To quote myself: "Invent a programming language that does not exist."

I've had this work both for "from scratch" descriptions of languages by providing grammars, and for "combine feature A from language X, and feature B from language Y". In the latter case you might have at least an argument. In the former case you do not.

Most humans struggle with tasks like this - you're setting a bar for LLMs most humans would fail to meet.

[skydhash]

As long as you create the grammar, the language exists. Same if you edit a previous grammar. You're the one creating the language, not the model. It's just generating specific instance.

If you tell someone that multiplying a number by 2 is adding the number to itself, then if this person knows addition, you can't be surprised if it tells you that 9*2 is 18. A small leap in discovery is when the person can extract the pattern and gives you 5*3 is 5+5+5. A much bigger leap is when the person discovers exponent.

But if you take the time to explain each concept....

[vidarh]

> As long as you create the grammar, the language exists.

Yes, but it didn't exist during training. Nothing in the training data would provide pre-existing content for the model to produce from, so the output would necessarily be new.

> But if you take the time to explain each concept....

Based on the argument you presented, nothing a human does is new, because it is all based on our pre-exististing learned rules of language, reasoning, and other subjects.

See the problem here? You're creating a bar for LLMs that nobody would reasonably assign to humans - not least because if you do, then "accusing" LLMs of the same does not distinguish them from humans in any way.

If that is the bar you wish to use, then for there to be any point to this discussion, you will need to give a definition of what it means to create something new that we can objectively measure that a human can meet that you believe an LLM can't even in theory meet, otherwise the goalpost will keep being moved when an LLM example can be shown to be possible.

[skydhash]

See my definition at : https://news.ycombinator.com/item?id=44137201

As mentioned there, I was arguing that without being prompted, there's no way that it can add something that is not a combination of the training data. And that combination does not act on the same terms that you would expect someone learning the same material would do.

In Linear regression, you can reduce a big amount of data to a small amount of factors. Every prediction would be a combination of those factors. According to your definition, those prediction will be new. For me what's new is when you retrospectively adds the input to the training data, find a different set of factors that gives you a bigger set of possible answers (generation) or narrows the definition of correct answers (reliability).

That is what people do when programming a computer. You goes from something that can do almost anything and you restrict it down to a few things (that you need). What LLM do is throwing the dice and what you get may or may not do what you want, and may not even be possible.

[vidarh]

That comment doesn't provide anything resembling a coherent definition.

The rest of what you wrote here is either also true for humans or not true for machines irrespective of your definitions unless you can demonstrate that humans can exceed the Turing computable.

You can not.