[IIAOPSW]

Here's an alternative to a model for truth. There is no truth, only power. Suppose we completely abandon logical semantics and instead focused on social semantics. Instead of the usual boolean True/False variables and logic relations, we'll have people valued variables and like/dislike relations. I system entirely for reasoning about the amount of pull and persuasion is present without ever circuiting down to any ground truth reasons. In other words, a bullshit reasoning system. Can ordinary truth reasoning be jerryrigged out of this system?

[seydor]

Yes, it s called empiricism

[IIAOPSW]

This was rhetorical. My point was that a system or model which cares about something other than the truth can, upon reaching a certain level of sophistication, be able to handle reasoning about truth. Eg, an AI that cares entirely about approval for what it says rather than the actuality of what it says could still end up reasoning about truth, given that truth is most heavily correlated with approval. I reject the premise that there has to be an a priori truth model under the hood.

[earthboundkid]

RLHF is basically just applying social power to the machine. It’s used for good (ChatGPT won’t help you spread Nazi memes) and hegemony (ChatGPT won’t help you overthrow capitalism).