[fzeroracer]

And I see people say this, that they've built a house but what they've built is something that structurally collapses immediately and fails due to problems that even an average person would've avoided had they learned how to build a house.

So did you build a house, or did you build a house?

[wvenable]

How can I argue about such a hypothetical?!? If I ask an LLM or my mother-in-law about how to get wine out of the carpet, and I get wine out of the carpet, maybe I have done it wrong and my house will burn down! Who knows?! Your argument is meaningless; there's nothing anyone can say to that.

[fzeroracer]

And if you ask an LLM about an effective cure for cancer and it spits out to drink bleach, are you going to follow it?

When you ask 'who knows' that's the point of research which was my original comment here. The same goes true for some random asshole telling you to drink bleach as it does an LLM, except people seemingly have hyped themselves into believing the LLM is more right somehow instead of being trained on every random asshole ever.

[wvenable]

> And if you ask an LLM about an effective cure for cancer and it spits out to drink bleach, are you going to follow it?

But it's not going to do that. It's literally designed to give the best and most accurate answer that it can. It's not perfect and I don't expect it to be perfect. According to my personal experience it's definitely good enough for a lot of things like recipes, coding, hobby stuff, and home repairs.

You're expecting me not to trust my own personal learned experience using AI as if somehow all my continued successes are worth nothing because you can invent some wild hypothetical. The more I use it, the more I understand its limitations and avoid them.

[somebehemoth]

> But it's not going to do that. It's literally designed to give the best and most accurate answer that it can.

This is not true. This is not how LLM's work. They have no concept of accuracy or "best".

[wvenable]

You missed my point.

The system designers (OpenAI, Anthropic, etc.) are absolutely trying to build a system that gives the best and most accurate answers possible.

The model itself does not have goals, intentions, or an internal concept of "best" or "accurate."

[fzeroracer]

> But it's not going to do that. It's literally designed to give the best and most accurate answer that it can.

No, no, no and no. This is the biggest mistake I see people consistently make with LLMs. It is not designed to give you the most accurate answer, it is designed to give you the most likely series of words following your prompt.

If an LLM is trained on 10 jackasses thinking bleach is a medicinal drink and 1 doctor who disagrees, it will by virtue of probability tell you to drink bleach. Companies add additional safeguards or system prompts to try and keep it 'on rails' but it's all probability based on what you prompt it. It is by the literal functionality of how it works to do so. A function depending on what said company ingests during training, many of which include the entire corpus of the internet.

If you do not understand how LLMs work under the hood then yes, you shouldn't trust your own personal learned experience because you've already demonstrated that you're wrong.

[wvenable]

> It is not designed to give you the most accurate answer, it is designed to give you the most likely series of words following your prompt.

That's not specifically true either; training is more complex than that. ChatGPT had to be trained, for example, to answer questions in a chat format.

> If an LLM is trained on 10 jackasses thinking bleach is a medicinal drink..

Again with the hypotheticals! You literally cannot discuss this subject without hallucinating things that don't exist. LLMs are trained on huge corpus of information from books to videos to reddit posts. Ultimately, statistically, it's going to predict the most common answer to something. Yes, that might be wrong but the vast majority of the time it's going to be right. And you know what, in the real non-hypothetical world, it works great. As much you don't want it to. You can hypothetically hallucinate as many weird unlikely scenarios as you want but that doesn't make it true.

[fzeroracer]

I see there is no point trying to get through to you. These are not hypotheticals, as an example just recently ChatGPT advised a user to mix bleach and vinegar. I have seen Googles AI hallucinate many details that I had to double check and would've been disastrous had I assumed it was truthy. There's a reason why models have the legal clauses like 'this is for entertainment purposes only' because they are trying to shield themselves from legal obligations when it makes catastrophic mistakes.

The people least willing to understand how the system works are also the most willing to blindly believe it.

[hodder]

Your arguments are much lower quality than an LLM will generate. You are so transparently biased against them it is shocking. I’d certainly rather learn from an LLM than someone like you who is both grossly misinformed and a dick about it.