[famouswaffles]

It's extremely easy to test arithmetic on random numbers it'll never have seen in the training set. GPT-4 can do arithmetic just fine.

[ResearchCode]

And it will give you some random number with maybe the right order of magnitude. Doesn't seem to do arithmetic very fine at all actually.

[famouswaffles]

Yeah that's not true lol. It's arithmetic is not perfect (mostly multiplication, addition is fine) but there's nothing random about even the wrong/non-exact numbers

[krainboltgreene]

we're talking about an llm dude

how am i supposed to have a conversation about someone who is gassing up "not perfect [arithmetic]" (something a wrist watch from the 80's can do) and won't even believe what the creators of said machine say about how it works