[kenjackson]

What does "novice" mean in this context? For example, I recently used chatGPT to explore file systems in the Linux kernel. I'm a novice in this field (but know Linux decently well, and the general field of file systems pretty well), but I could pretty easily tell the information it was giving me was correct.

[michelledepeil]

When you don't have enough background knowledge to even verify whether the answer is plausible. Or when you don't know what question to ask.

It may very well be that newer AIs and models even manage to get that right more often, though. Or that a canny prompter will start by asking which question to subsequently ask.

As a side note, you're not a novice if you know the general field of file systems pretty well, IMHO.