[gchadwick]

> ChatGPT can just send you something that is completely wrong, and you have no way of knowing.

This is true, if you decide to take a ChatGPT answer at face value without any further work. Personally I find it useful sometimes to ask an LLM a question, get an answer and the verify that answer for myself. Doing web searches and pulling together relevant information to get the answer for a question can be harder than getting an answer and then looking to verify it. Perhaps something like that was going on here, impossible to know of course.

[neuroelectron]

ChatGPT rarely gives you sources for anything outside of writing software and doing homework

[lionkor]

Here's an example: When asked about path buffer length in a programming context, ChatGPT 4-o claimed today tht 256 bytes is sufficient for *most systems*. That's an entirely false claim, like, completely invalid. It only says this because that's the tone that is expected of it. You can clearly tell that the info it wanted to convey was "256 is sufficient [here]" but it LOVES just making things sound more general than they are.

you aren't gonna look up if that little detail is right; you're gonna slowly absorb more and more subtly false info.

[red-iron-pine]

the point of it is that I don't have to check. otherwise now i've just added an extra set of typing and validation.

plus, now i've been biased by the immediate response. if it says "these CVEs don't have vulnerabilities" then I'm now thinking they're probably okay and just need to validate, instead of starting from zero and doing due diligence. this will lead to confirmation biases or laziness.