Hacker News new | ask | show | jobs
by ToValueFunfetti 613 days ago
>Whether that sequence matches any kind of objective truth is orthogonal to how models work.

Empirically, this cannot be true. If it were, it would be statistically shocking how often models coincidentally say true things. The training does not perfectly align the model with truth, but 'orthogonal' is off by a minimum of 45 degrees.
1 comments
viraptor 613 days ago
It matches the training data. Whether the training data matches truth (and whether it's correctly understood - sarcasm included) is a completely separate thing.

> The training does not perfectly align the model with truth, but 'orthogonal'

Nitpicky, but the more dimensions you have, the easier it is for almost everything to be orthogonal. (https://softwaredoug.com/blog/2022/12/26/surpries-at-hi-dime...) That's why averaging embeddings works.

link
ToValueFunfetti 613 days ago
I went to school to learn about the world and the overwhelming majority of that learning was from professors and textbooks. Whether the professors' beliefs and the textbooks' contents reflected the true properties of the world was a completely separate thing, entirely outside of my control. But I did come away with a better understanding of the world and few would say that education is orthogonal to that goal.

If you add two vectors that don't have a truth component (ie. are orthogonal to the truth), the resulting vector should be no closer to the truth. If you start with random weights and perform some operation on them such that the new weights have a higher likelihood of producing true statements, the operation must not have been orthogonal to the truth. Am I wrong there?

link
viraptor 613 days ago
> But I did come away with a better understanding of the world and few would say that education is orthogonal to that goal.

That's due to the reward function / environment. But even outside extremes like North Korea, lots of education environments value conformity over independent analysis.

link
ToValueFunfetti 613 days ago
Certainly an AI trained on North Korean data would emerge with some very suspect beliefs regarding Kim Jong-Un. My point is just that aligning something with training data is aligning it with truth, to the degree that the training data is true and regardless of why it is true. educate(me, truth) can hardly be called orthogonal to the truth, even if the 'educate' and 'me' terms do nothing to prevent educate(me, falsehood).
link
timcobb 613 days ago
Isn't this the same thing that happens when you train a human on truths vs falsehoods?
link