[airstrike]

Makes sense, but I don't know why they'd let said prompt voodoo touch RL. I'm OK with prompting to get the model to, I don't know, write better Rust or build Excel spreadsheets. I am less OK with making it "quirky" or having some "personality" in a way that becomes ingrained in the model for everyone else

TL;DR the cringe nerdy shit should be (optionally) switched on at inference, not as part of RL

[nearbuy]

They do it because training different personalities is more effective than just changing the system prompt. Ever try asking ChatGPT to adopt a specific personality in a prompt? Its standard style bleeds through.

As the article says, the personalities weren't supposed to affect other personalities. OpenAI was as surprised by the goblins as you are. Training can be tricky.