[esalman]

I've seen stuff like that too. I asked ChatGPT a tricky geometry question and it got a ratio reversed and ended up producing wrong answer.

I have also tried to use ChatGPT like Google search at the beginning by asking short, curt questions, and got underwhelming results.

The trick is not to try to trick ChatGPT but pretend that you are an idiot and it is a very sophisticated colleague who just needs step by step details. Then you can also generate some of the seemingly amazing results people are getting. For example yesterday I used it to scrape physician information from my insurance website, followed by their Google ratings and so on.