What strikes me about ChatGPT is the blatantly wrong answers it can give. I asked ChatGPT to solve a augmented matrix using gaussian elimination, and it failed in this straightforward task spectacularly.
Perfect example of "confident sounding hallucinations", I was just googling (a moment ago) why olive oil caused a burning sensation. It turns out there's a substance called oleocanthol[1] and there are receptors for that mainly in the throat. But while googling it, I see on Quora (which made it to the previews on Google) an "Assistant - Bot" response that is completely wrong: "Drinking olive oil can cause a burning sensation in the back of your throat due to its high fat content[...]"[2]
It wouldn't be notable if someone specifically asked ChatGPT, knowing its limitation, but using it to automatically populate Quora and Google with it is pretty bad. People are using LLM to fill the web with BS.
One is dumb as a brick, then other isn't. If you don't specify, then your comment should be dismissed out of hand.
Also, it's a well-known limitation of all current LLMs that they're terrible at basic algebra. Instead of trying to replace BLAS or Maple with it, ask it to write the Python code or produce the Mathematica expression.
If you subscribe to ChatGPT+ you can just ask GPT-4 to verify the approach by doing a web search, that often works quite well.