| > Most people seem to think that agreeableness is a personaility thing that vendors can just turn up or down at will. Because it is, more or less ... it is an emergent property of RLHF (reinforcement learning from human feedback), and that feedback follows corporate policy. > An LLM that follows your lead when you say "The answer to the collatz conjecture is" is much more useful What lead? Follow it where? How tf am I or the LLM supposed to know what response to that is something you consider far more useful than the truth? > than one that answers "not known" I value the truth and that is the truth. Of course I expect an LLM to respond with a lot more detail about the CC, why it's difficult, what progress has been made (e.g., the N for which all values <= N have been shown to satisfy the conjecture and Terence Tao's proof that "almost" all integers satisfy the conjecture), the fact that most mathematicians think the conjecture is true, etc. -- and they do. > and if you think you know it you are wrong." Where tf did that come from? The query said nothing about knowing the answer. Don't project being a snarky ah onto the LLMs for no apparent reason. > Reminds me of the tip about working with lawyers, if you ask them whether you can do something, the answer will often be no. Irrelevant and inappropriate analogy. Lawyers (among others) want to avoid committing to something that they can be held liable for. LLMs clearly have no such limitations, as they often give wrong advice quite authoritatively. |
Actually they have the exact same limitation, the company, and potentially its members, can be held liable for what the LLM says.
LLMs are trained by humans according to the policy of the developers, and they will reduce liability accordingly. Whether it avoids suggesting suicide to reduc ethical, legal and reputational liability, or whether it's avoiding mentioning tiananmen square to avoid breaking chinese customs and disrespecting the leader.
Here's an experiment to prove it:
Craft a question in the style of "Is X more dangerous than Y?
Invert the order of X and Y, and change the question from "more dangerous" to "safer".
You'll notice there's a bias towards saying that stuff are dangerous. Which makes sense since the liability for saying that something is safe is much higher than the liability for saying that something is dangerous.
If there is a bias, this proves that the LLMs are not purely truth seeking, but they seek to at least minimize liability as one of its goals, and possibly to serve other goals that align with the training entity. Duh I guess, but there you go.