[jqpabc123]

Q: How many letters are in the word "intelligence"?

A: The word "intelligence" contains 11 letters: i-n-t-e-l-l-i-g-e-n-c-e.

Wrong! The correct answer is 12.

This is definitely not AGI! It is a crude statistical attempt that some people confuse with AGI.

[alphan0n]

>Prompt: Use your stateful python environment to count the letters in this comment: “Q: How many letters are in the word "intelligence"? A: The word "intelligence" contains 11 letters: i-n-t-e-l-l-i-g-e-n-c-e. Wrong! The correct answer is 12. This is definitely not AGI! It is a crude statistical attempt that some people confuse with AGI.”

>Show your work. Then generate a Jupyter notebook that performs the calculations so I can double check, provide a link to download it from my storage when complete.

>Answer: The total number of letters in the provided comment is 187.

>You can download the generated Jupyter notebook to review and verify the calculations using the link below:

>Download the Jupyter Notebook

https://chatgpt.com/share/6759dd5a-140c-8011-b132-5664f3aaa4...

[alphan0n]

Just an aside, this was pretty impressive in my opinion.

I don’t know what AGI really means at this point, and I don’t think this is quite there, but this is an example of ChatGPT (4o) choosing the correct context (counting only alphabet characters not total character count), parsing the comment correctly (non-trivial given the usage of delimiters in both the prompt and comment and reference to counting letters in the comment not being interpreted as a part of the instructions), choosing the correct python tools, arriving at the correct answer, then generating and populating a working, downloadable, notebook to check the work. All in one reply.

It looks like you would need to continue the conversation from the link I provided then regenerate the prompt/reply to see the processing blocks and download the notebook.

[adastra22]

Again, you need to define AGI. According to the classical definition of the term by the people who coined it, the example you gave proves that it is AGI. You were able to specify a problem using a fully generic interface (chat) that the AI was not specifically trained on, and it got an answer close to the right answer. The process it used to generate that answer is general intelligence.

[jqpabc123]

Semantics aside, it is definitely lacking and leaves much to be desired in terms of "intelligence".

Either it didn't understand the question or it can't count --- take your pick. Any results come at a high cost and are simply not reliable.

[adastra22]

What ChatGPT actually sees when you input that question is the output of the tokenizer:

[5299, 1991, 18151, 553, 306, 290, 2195, 392, 491, 33465, 69029]

This happens to be 11 tokens, but I think that's a coincidence. Token 491 is "int" and token 33465 is "elligence", but ChatGPT doesn't actually see the letters.

How can you expect it to count, given those limitations? It had to guess how many letters each token represented. It got close, but not exact.

This is an artificial example pretty much maximally designed for ChatGPT to screw up.

[jqpabc123]

This is an artificial example pretty much maximally designed for ChatGPT to screw up.

This is an extremely simple example that ChatGPT screws up maximally.

If it can't be trusted for a simple, obvious example, why should it be trusted in less obvious cases were accuracy and reliability are important.

Answer --- it shouldn't. Maximal cost with minimal reliability.

[adastra22]

I explained why, but you seem to be attached to an anti-AI belief, so I don't think it is worthwhile to continue further.

[jqpabc123]

I don't care why --- and neither will any potential application that demands a valid, affordable solution.

The IRS don't care why your tax return was wrong, they still charge a penalty.

*Why* doesn't excuse the basic facts --- that it is simply not reliable and is very expensive.

[alphan0n]

Seems like it can count just fine: https://news.ycombinator.com/item?id=42391312

Maybe you’re using it wrong.