| When their precision mismatches their accuracy (or your expectations as driven by their design), just like with any other metrology tool. Now you might say: "but the datasheet will give you the tolerances, and the manual will tell you to mind it!" And yes, that's true. Just like how LLM providers also do: they tell you that outputs may be arbitrarily wrong, and that you should always check for mistakes. Is this bullshit? Yes. So are metrology tools that have a mismatching precision and accuracy, need calibration, and have designs that fail to make you mind either of these, sending you to reading duty instead. Which just so happens to be a whole lot of them. It is also absolutely not bullshit of course, because it is a fundamental limitation, just like those properties are for metrology devices. LLMs produce arbitrary natural language. Short of becoming able to perfectly read and predict the users' mind, they'll never be able to make any hard assurances, ever. Defective devices also exist, and so do incorrect / incomplete documentation. |