[SOLAR_FIELDS]

This is such a technopurist take. People who use LLM’s already know they can give wrong information. Your documentation won’t be able to cover every single possible contextual scenario that an LLM can help with. I think there are valid reasons to not allow OpenAI to spider you, but this is just a really silly one that feels pretty egotistical. People aren’t going to this guy saying “well OpenAI said your software works this way and it doesn’t”. It’s an entirely contrived scenario that doesn’t exist in reality.

[EatingWithForks]

> People who use LLM’s already know they can give wrong information

I think this is unfortunately much less true than expected... Lawyer using chatgpt.. teachers using chatgpt... even professors using chatgpt... as if its a source of truth.

[SOLAR_FIELDS]

There have been a few instances, sure, and they made headlines, but that was pretty early on when LLM behavior was not well understood. I think that fake citations (as the most obvious and well documented example) are a well understood problem now, and if you google “ChatGPT fake citation” you only get a few articles mostly referencing the same couple of cases from months ago. It doesn’t seem pervasive at all.

[plufz]

Anecdotal, but everytime I tell someone that the citations from ChatGPT can be bogus they are very surprised. They know that the answers can be incorrect, but they don't understand the process behind an LLM well enough to understand that a citation can be generated in the same way the rest of the text is.

[frizlab]

My CTO showed me oh so ever happy that he translated in English something I did in French to send it to a foreign corporation. I read it, the first word was wrong. Most of the rest had more or less the same meaning, but not that first word. I argued ChatGPT is dangerous because he was gonna send an incorrect document because of it (he had not sent it yet), but he straight up _refused_ to admit the word was wrong and saying “the meaning is almost the same!” Well it was not… So yeah some people are not aware ChatGPT can be wrong/dangerous, and some people are worse, and refuse to believe/listen to actual people and prefer a robot.

[dwaltrip]

It sounds like he was excited about it using some new tech and then was upset when you blithely smashed on him.

Would it have been more emotionally mature of him to put that aside and listen to your criticisms? Yes, of course. But you probably could have saved some trouble and conflict by sharing in his joy a little before helping him understand the pitfalls and issues.

[gavinhoward]

They're only not doing that because my software is not common yet. But look at GitHub issues for any semi-famous project, and you'll see a lot of questions about misunderstandings, and that's before LLM's poisoned everything.

[davely]

> But look at GitHub issues for any semi-famous project, and you'll see a lot of questions about misunderstandings

This usually happens because people don’t read documents to understand why something isn’t working in the first place or the documentation is not clear.

If anything, an LLM makes this sort of stuff more accessible.

Anecdotally, I find using something like ChatGPT to rubber duck engineering problems with various libraries to be much more enjoyable and useful than going to Stack Overflow or mucking through overly verbose (or not verbose enough) docs.