[jqpabc123]

Semantics aside, it is definitely lacking and leaves much to be desired in terms of "intelligence".

Either it didn't understand the question or it can't count --- take your pick. Any results come at a high cost and are simply not reliable.

[adastra22]

What ChatGPT actually sees when you input that question is the output of the tokenizer:

[5299, 1991, 18151, 553, 306, 290, 2195, 392, 491, 33465, 69029]

This happens to be 11 tokens, but I think that's a coincidence. Token 491 is "int" and token 33465 is "elligence", but ChatGPT doesn't actually see the letters.

How can you expect it to count, given those limitations? It had to guess how many letters each token represented. It got close, but not exact.

This is an artificial example pretty much maximally designed for ChatGPT to screw up.

[jqpabc123]

This is an artificial example pretty much maximally designed for ChatGPT to screw up.

This is an extremely simple example that ChatGPT screws up maximally.

If it can't be trusted for a simple, obvious example, why should it be trusted in less obvious cases were accuracy and reliability are important.

Answer --- it shouldn't. Maximal cost with minimal reliability.

[adastra22]

I explained why, but you seem to be attached to an anti-AI belief, so I don't think it is worthwhile to continue further.

[jqpabc123]

I don't care why --- and neither will any potential application that demands a valid, affordable solution.

The IRS don't care why your tax return was wrong, they still charge a penalty.

*Why* doesn't excuse the basic facts --- that it is simply not reliable and is very expensive.

[alphan0n]

Seems like it can count just fine: https://news.ycombinator.com/item?id=42391312

Maybe you’re using it wrong.