One of the reasons it's useless is that if there's no way to do exactly what you're asking, ChatGPT will just invent a way. I've wasted a bunch of time trying to find docs for functions ChatGPT claims exist, only to conclude that it made them up.
ChatGPT is like a smart human inventing a hypothetical API which would solve your problem. A service like StackOverflow has people who are actually familiar with what actually exists in reality. The human will usually know enough to come up with some alternative approach or workaround where ChatGPT hallucinates an answer.
It’s like when you go looking for an answer in GitHub’s issues discussion, and while scanning down you see a code snippet with a perfectFunctionForThis()… only to realize a few moments (or longer) later the commenter is suggesting it would be nice if this function existed and if this pattern could work.
Sounds solvable though once LLMs like ChatGPT become "live" (e.g connected to the internet) they can perhaps verify their answers. They will probably have their own development environment at some point.
So these things might still happen in the future but rarely.
That's because it's still an early version. Also, it's trained on data over 2 years old. OpenAI needs to figure out how to regularly refresh its training. Newer and better models for programming should also help.
The info being old is an excuse for not knowing about recent stuff. It's not an excuse for hallucinating answers to questions it doesn't know the answer to (or for which no answer exists).
I have seen nothing to suggest that future modes will fix the hallucination problem. I wouldn't be surprised if it's just an inherent, unfixable issue with the concept of neural network language models – they don't have a model of the world from which they produce logically coherent text, they simply have a giant statistical model for which words tend to occur in which order – so I don't see why we should expect a language model to not hallucinate. But I'm not an expert on the current state of the art, so if you have some links to papers which indicate that the language model hallucination problem is solvable, I'm open to admitting I may be wrong.
ChatGPT may not feel outright toxic but easily devolves into a buttheaded condescending nanny if you try to push some of it's "moral" boundaries. I tried to have it help me come up with ideas for a programming language that would be used to spite employers and or coworkers. Boy did I get a lesson about my negative attitude.
You can tell ChatGPT to reply in the style of someone else, or even take on another role, which makes its responses differ. It's interesting that it even has moral boundaries. This could be part of safe AI, which a lot of people have complained about long before ChatGPT launched.
ChatGPT is like a smart human inventing a hypothetical API which would solve your problem. A service like StackOverflow has people who are actually familiar with what actually exists in reality. The human will usually know enough to come up with some alternative approach or workaround where ChatGPT hallucinates an answer.