[fnordpiglet]

They gave you all the information you needed to ask the next question. It specified “in a closed system,” which meant it made an assumption of the closed system and told you. An observant reader would ask then “and what about an open system?” Or if they’re not that sophisticated, “what about a non-closed system?”

You’ve not talked to many theoretical physicists I see. I’ve not noticed they’re usually that good at explaining things, especially to lay people. They tend to be very good at talking with people at the same level or more advanced, usually in a mathematical form. They tend to be pretty bad at imitating a human.

[mtlmtlmtlmtl]

Yeah, if you understand the field or are observant enough, you can tell the answer is fishy. And if you don't, you can't tell.

So what, anyone who gets a wrong idea from ChatGPT is just unsophisticated and we should ignore it? Why are you so incredibly set on invalidating any criticism of ChatGPT?

You don't see a problem with advertising this LLM as something it isn't? Lots of people seem willing to take ChatGPT completely at face value now, and walk away having learned a bunch of nonsense. And lots of them are smart people, they've just been duped by the hype into thinking LLMs can do things they fundamentally can't.

[fnordpiglet]

Oh I think it should be improved for sure. I just think this is a bad example. I think most of the fact checking can be done using any modern information retrieval system and you can build algorithms that will regenerate answers until they’re factually correct, or use the IR to hint the answer to correctness. We also have very powerful semantic inference engines and other tools that complement LLM output. I think judging the possibilities by the beta is simplistic, and folks are unfairly down on the achievement by picking nits.