I think the difference, right now at least, is that people will go, "well, I'm not sure about this so I think we should look it up, but this is what I think" - the AI doesn't do that. It lies in the same exact way it tells truths. How are you supposed to make decisions based off of that information?
Sure, it's not lying, you're right, there's no will there, I'm anthropomorphism. It is producing entirely wrong facts / pseudo-opinions (as it can't actually have an opinion).
I was about to suggest "pathologically dishonest", but then I looked up the term and that seems to require being biased in favour of the speaker and knowing that you're saying falsehoods.
"Confabulate" however, appears to be a good description. Confabulation is, I'm told, associated with Alzheimer's, and GPT's output does sometimes remind me of a few things my mum said while she was ill.
I was too vague I think. The only place where I can see it being acceptable right now is code because I have a whole other system that will call out failures - I can rely on my IDE and my own expertise to hopefully catch issues when they appear.
Outside of the code use case, what should I rely on ChatGPT for that won't have me also looking for the information somewhere else? I suppose subjective soft things, like writing communications. But I can't rely on it for information.
Again, the idea that you should rely on any single source for information is the issue. Nothing changes with ChatGPT other than the apparent expectation that it is infallible.
So what are the use cases that I would use ChatGPT to find information that would speed up my work but would still require me to verify the information? If nothing changes with ChatGPT what is its use as a tool (assuming you want to use it to get information)?
Sometimes it does, but I asked ChatGPT (not 4) to give me song lyrics for a song that it should have had data for and it gave me entirely wrong lyrics. I asked again and it gave me me more bad lyrics, not even close, and it didn't even pretend it didn't know, the lyrics would have been convincing. If I didn't already know the material I wouldn't know it was confabulating.