[charcircuit]

>I mean, is there a data point in the dataset used to train where you can read 2241 + 19873 = 22114? Quite unlikely...

But there might be something like xxx1 + xxxx3 = xxxx4 in the dataset so it can learn the pattern.

[qumpis]

That's the astonishing bit

[williamtrask]

It really isn’t. You see a lot of things when reading 500 billion tokens

[chaxor]

Yeah, I'm *totally* unimpressed.

I, for one, learn all my math without ever seeing any math or logic examples at all.

"Teacher, what is this '34+12' stuff - I've already developed a complete grand unification theory on my own - I don't need examples of what you call 'addition'" - apparently everyone unimpressed by nlp today

[drdeca]

they didn't mean that it was astounding that something of the form "xxx1 + xxxx3 = xxxx4" was in the training set, but that it managed to "learn the pattern".

[amelius]

They should have asked questions like:

What is twothousandfortyone plus nineteenthousandeighthundredseventythree?