[ijk]

I asked, just now:

> How many 'r's are in strawberry?

> ChatGPT said: The word "strawberry" has 2 'r's.

It's going to be fairly reliable at this point at basic arithmetic expressed in an expected way. That's pretty baked in. Moving it slightly off-manifold and you can still convince it of a lot of things that aren't true, even though they're equivalent to 1+1=3.

[commakozzi]

It hasn't incorrectly answered this question in a very long time. Something tells me you're being dishonest to try to make a point.

[ijk]

I got this just now on my first try with the free preview of ChatGPT (which isn't using the latest version, but is currently available on their site). I was surprised, I expected to have to work harder for it to fail like that.

[lucumo]

This triggered me to retest. Let me first apologize for calling you a liar. It's possible that you saw this.

I tried your formulation on the ChatGPT homepage in incognito mode (to rule out personalization for me). It said 2, so it's possible you saw that as well. I tried the same experiment again (new incognito mode, same phrasing) and it said 3.

That made me curious if phrasing made a difference and how often it would go wrong.

I tried 5 times with my own phrasing ("So, how many Rs in strawberry?") and got 5 times 3 as an answer.

I tried 5 times with your phrasing ("How many 'r's are in strawberry?") and also got 5 times 3 as an answer.

So 2 is an answer that can be given, but seems to be rare. At least in my tests.

[lucumo]

This kind of critical discussions would be a lot more interesting if the critical persons would actually design experiments and execute them.

But sure, I'll try this for you. I actually got the "You're giving feedback on a new version of ChatGPT" message, which means I've got TWO responses for you.

--- Response 1

Me: So, how many Rs in strawberry?

ChatGPT: There are three R's in strawberry:

* stʀawbeʀʀy

(One at position 3, and a double dose at the end. R-rich!)

--- Response 2

There are three R's in strawberry.

Count ’em:

* s

* t

* r

* a

* w

* b

* e

* r

* r

* y

The R’s are at positions 3, 8, and 9. (And now I can’t stop hearing it pronounced “strawbuh-RRR-y.”)

---

So now it has proven you a liar twice.