[wildmanx]

I'd like to see doing this with random people on the street and then compare performance. You may be surprised.

[NateEag]

Not really.

Average person on the street is going to correctly say "geez, I dunno. Can I use my phone?"

If you don't forbid them to, then they'll whip it out and get 98% correct (I figure they'll typo a few).

This model didn't have enough understanding to do that (since it literally has no understanding at all).

[wildmanx]

I don't know where you live, but 98% correct is not what would happen around here.

Edit: Oh, 98% _with_ a calculator. What if you force them to do it by hand?

[ceejayoz]

> What if you force them to do it by hand?

Why?

GPT-3 isn't doing it that way. The importance is understanding the question, not being able to do math in your head.