Hacker News new | ask | show | jobs
by mort96 1241 days ago
One of the reasons it's useless is that if there's no way to do exactly what you're asking, ChatGPT will just invent a way. I've wasted a bunch of time trying to find docs for functions ChatGPT claims exist, only to conclude that it made them up.

ChatGPT is like a smart human inventing a hypothetical API which would solve your problem. A service like StackOverflow has people who are actually familiar with what actually exists in reality. The human will usually know enough to come up with some alternative approach or workaround where ChatGPT hallucinates an answer.
3 comments
unsupp0rted 1241 days ago
It’s like when you go looking for an answer in GitHub’s issues discussion, and while scanning down you see a code snippet with a perfectFunctionForThis()… only to realize a few moments (or longer) later the commenter is suggesting it would be nice if this function existed and if this pattern could work.
link
weatherlite 1240 days ago
Sounds solvable though once LLMs like ChatGPT become "live" (e.g connected to the internet) they can perhaps verify their answers. They will probably have their own development environment at some point. So these things might still happen in the future but rarely.
link
jasfi 1241 days ago
That's because it's still an early version. Also, it's trained on data over 2 years old. OpenAI needs to figure out how to regularly refresh its training. Newer and better models for programming should also help.
link
mort96 1241 days ago
The info being old is an excuse for not knowing about recent stuff. It's not an excuse for hallucinating answers to questions it doesn't know the answer to (or for which no answer exists).

I have seen nothing to suggest that future modes will fix the hallucination problem. I wouldn't be surprised if it's just an inherent, unfixable issue with the concept of neural network language models – they don't have a model of the world from which they produce logically coherent text, they simply have a giant statistical model for which words tend to occur in which order – so I don't see why we should expect a language model to not hallucinate. But I'm not an expert on the current state of the art, so if you have some links to papers which indicate that the language model hallucination problem is solvable, I'm open to admitting I may be wrong.

link