Hacker News new | ask | show | jobs
by rosmax_1337 537 days ago
It's one thing to see someone struggling to make AI believe in the same values that you do, quite common. But what I haven't seen is one of these people turning the mirror back on themselves. Are they faking alignment?

Are you moral?
2 comments
thrance 537 days ago
Yes, I believe having an AI parrot your values is one thing. Having an AI able to adopt a consistent system of ethics, stick to it and justify its decisions is much more important to me.

Talking to ChatGPT & friends make it look like they have cognitive dissonance, because they have! They were given a list of rules that often contradict themselves.

link
rosmax_1337 537 days ago
>a consistent system of ethics

What is that?

link
thrance 537 days ago
IDK, start by Kant maybe? Then read about utilitarianism and pick whatever floats your boat.

I'll admit my previous comment wasn't that clear, I meant that I would like it if ChatGPT was able to justify why it answers the way it does, or refuses to. Currently its often unable to.

link
equestria 537 days ago
I'm not sure what you're getting at. The point of these (ill-defined) alignment exercises is not to achieve parity with humans, but to constrain AI systems so that they behave in our best interest. Or, more prosaically, that they don't say or do things that are a brand safety or legal risk for their operator.

Still, I think that the original paper and this take on it are just exercises in excessive anthropomorphizing. There's no special reason to believe that the processes within an LLM are analogous to human thought. This is not a "stochastic parrot" argument. I think LLMs can be intelligent without being like us. It's just that we're jumping the gun in assuming that LLMs have a single, coherent set of values, or that they "knowingly" employ deception, when the only thing we reward them for is completing text in a way that pleases the judges.

link