[krainboltgreene]

It statistically outputs the next probable value of text. There is a lot of math related text in the CommonCrawl (likely the biggest source of it's training). That is all the company who owns it claims that it does. There is no calculation, therefor no basic arithmetic being done correctly.

[kgeist]

If it's only statistics, what is the neural network for?

[kgwgk]

Implement the probabilistic model of token sequences and fit it to the training data?

You could say the same for a character recognition system.

[jackmott42]

emergent behavior does happen with neural networks. It is correct to say that GPT 4 doesn't do math correctly, but it may be incorrect to say a large language model can't in principle.

[roflyear]

Is there any evidence it can? It's not incorrect to say there is no God for example

[famouswaffles]

It's extremely easy to test arithmetic on random numbers it'll never have seen in the training set. GPT-4 can do arithmetic just fine.

[ResearchCode]

And it will give you some random number with maybe the right order of magnitude. Doesn't seem to do arithmetic very fine at all actually.

[famouswaffles]

Yeah that's not true lol. It's arithmetic is not perfect (mostly multiplication, addition is fine) but there's nothing random about even the wrong/non-exact numbers

[krainboltgreene]

we're talking about an llm dude

how am i supposed to have a conversation about someone who is gassing up "not perfect [arithmetic]" (something a wrist watch from the 80's can do) and won't even believe what the creators of said machine say about how it works