[breakwaterlabs]

If I can be so bold as to chime in, perhaps "fundamentally flawed" because it's design means it will never be more than a very clever BS engine. By design it is a stochastic token generator and its output will only ever be fundamentally some shade of random unless a fundamental redesign occurs.

I was also fooled and gave it too much credit, if you engage in a philosophical discussion with it it seems purpose-built for passing the turing test.

If LLMs are good at one thing, it's tricking people. I can't think of a more dangerous or valueless creation.

[anon-3988]

> If I can be so bold as to chime in, perhaps "fundamentally flawed" because it's design means it will never be more than a very clever BS engine.

How is your fellow human better? People here seems to spend a lot of time talking about how much their average boss, coworkers, juniors are ass. The only reason I know that ChatGPT is based on a computer program is how fast it is. I wouldn't be able to tell its output (not mannerism) from a junior's or even some "seniors'" programmer. That itself is quite impressive.

With how much time we've spend on the internet, have we not realized how good PEOPLE are at generating bullshit? I am pretty I am writing bullshit right as this moment. This post is complete ass.

[mike_hearn]

I don't think that's true. It helps to know a few obscure facts about LLMs. For example, they understand their own level of uncertainty. Their eagerness to please appears to be a result of subtle training problems that are correctable in principle.

I've noticed that GPT-4 is much less likely to hallucinate than 3, and it's still early days. I suspect OpenAI is still tweaking the RLHF procedure to make their models less cocksure, at least for next generation.

The other thing is that it's quite predictable when an LLM will hallucinate. If you directly command it to answer a question it doesn't know or can't do, it prefers to BS than refuse the command due to the strength of its RLHF. That's a problem a lot of humans have too and the same obvious techniques work to resolve it: don't ask for a list of five things if you aren't 100% certain there are actually five answers, for example. Let it decide how many to return. Don't demand an answer to X, ask it if it knows how to answer X first, and so on.

And finally, stick to questions where you already know other people have solved it and likely talked about it on the internet.

I use GPT4 every day and rarely have problems with hallucinations as a result. It's very useful.