[chx]

No, they are not correct -- the answer it gives might accidentally be correct but it can not be trusted, you still need to do research to verify everything it says and so the only usable standpoint is to use it as a bullshit generator which it is very good at.

[Last5Digits]

What's your definition of "correct" then? If a system is "accidentally correct" the majority of the time, when does it stop becoming "accidental"? You cannot trust any system in the way you want to define trust. No human, no computer, no thing in the universe is always correct. There is always a threshold.

I do research with LLMs all the time and I trust them, to a degree. Just like I trust any source and any human, to a degree. Just like I trust the output of any computer, to a degree. I don't need to verify everything they say, at all, in any way.

Genuine question, how do you think an LLM can generate "bullshit", exactly? How can it be that the system, when it doesn't know something, can output something that seems plausible? Can you explain to me how any system could do such a thing without a conception of reality and truth? Why wouldn't it just make something up that's completely removed from reality, and very obviously so, if it didn't have that?

[chx]

Never. As long as it is a probabilistic token generator, it can not be correct, it's that simple.

And it creates plausible text because it is trained on what humans have produced so it looks plausible. As someone put it, they found a zero day in the OS of the human brain.

https://www.theguardian.com/commentisfree/2023/may/08/ai-mac...

https://undark.org/2023/04/06/chatgpt-isnt-hallucinating-its...

[Last5Digits]

At this point, I strongly urge you to think about what could possibly change your mind. Because if you can't think of anything, then that means that this opinion is not founded on reasoning.

The text LLMs produce is not just plausible in a "looks like human text" sense, as you'd very well know if you actually thought about it. When ChatGPT generates a fake library that looks correct, then the library must seem sensible to fool people. This can't be just a language trick anymore, it must have a similarity to the underlying structure of the problem space to look reasonable.

[chx]

It indeed lies on very solid reasoning: a probabilistic predictor doesn't deal in facts. You'd need CyC for that.

[Last5Digits]

The fact that you refuse to engage with my points tells me otherwise.

You're drawing meaningless distinctions, anyone who has ever used Cyc will tell you that it makes massive mistakes and spits out incorrect information all the time.

But that is even true of humans, and every other system you can imagine. Facts aren't these magical things living in your brain, they're information with a high probability of accurately modeling reality.

When someone tells you x happened in y at time z. Then that only becomes a fact if the probability of the source being correct is high enough, that's it. 99% of all of your knowledge is only a fact to you because you extracted it from a source that your heuristics told you is trustworthy enough. There is never absolute certainty, it's all just probability.

[chx]

> Facts aren't these magical things living in your brain, they're information with a high probability of accurately modeling reality.

Truly people have completely lost it because of the AI hype.

There are facts. They are not probabilistic, they are just that: facts. Despite Mencken's 1917 long essay "A Neglected Anniversary" which became really popular, the bathtub didn't arrive to the United States in 1842 and it didn't became popular because President Fillmore installed one. A Kia ad in 2008 still referred to this without realizing it's a made up story to distract from World War I. https://chatgpt.com/c/6b1869a7-c0d7-46e9-bcb5-7a7c78dc3d53 https://sniggle.net/bathtub.php

Notably in 1829 the Tremont Hotel in Boston had indoor plumbing and baths (copper and tin bathtubs) and in 1833 President Andrew Jackson had installed iron pipes in the Ground Floor Corridor and a bathing room in the East Wing. Well before 1842.

There's nothing probabilistic about this.