I suspect most people on this forum can do arithmetic for any "reasonable" size number. It might take weeks to complete, but most people on this forum can calculate large numbers by hand.
Post moving. "Reasonable" is just an arbitrary line. Especially since most if not all would make some mistake somewhere along the line.
You can greatly increase GPT's arithmetic capabilities tackling it like a problem to solve "on paper" in context. And this was done on 3.5 not 4.
https://arxiv.org/abs/2211.09066
If its going to take weeks, most people will get it wrong. That's a lot of calculations to never get wrong and never misinterpret some prior note you left
Okay, but we have since invented machines that can do arithmetic correctly, every time. When we try to do maths via an LLM, we're just throwing all of that away.
So ? I didn't tell you to use GPT-4 for arithmetic over a calculator. I simply pointed out that the only standard where GPT-4 is not good at arithmetic is a standard humans wouldn't fit the bill either. Especially since zero shot "mental" arithmetic is not even close to GPT-4 at its most accurate.
The discussion started "what would it take to convince people that [insert favourite LLM] is good at maths", and the response to that IMHO is that we have much better tools to do arithmetic (I don't even want to say maths), even if humans themselves are also poor at arithmetic.
What's the point of building a system to be equally bad as humans at something that we know humans are bad at? LLMs have their uses but (at least at the current stage) performing arithmetic calculations is not one of them (to say nothing of more advanced mathematics).