[bell-cot]

For basically the same reason that ELIZA ( https://en.wikipedia.org/wiki/ELIZA ) couldn't count to a million. It's doesn't actually understand things, and it's not a general intelligence.

In human terms - think of ChatGPT as a very well-born person, sadly with profound brain issues, who you are meeting at a fancy cocktail hour. His family has spent years and $millions getting him to the point of looking & sounding pretty good, within that very limited social context. But he's never actually made a PB&J sandwich. Let alone gotten dressed and gone to the grocery store by himself.

Over cocktails, Mr. ChatGPT seems darned impressive. But that's because your "is this guy mentally fit?" instincts - developed and tuned in interactions with normal people - naturally jump to flawed conclusions.