Hacker News new | ask | show | jobs
by airstrike 52 days ago
That all of their model outputs should be influenced by whatever personality prompt voodoo the wise artisan at OpenAI decided to stuff it with during RL should give everyone pause.

That Nerdy personality prompt made me gag. As a card-carrying Nerd, I feel offended
2 comments
nearbuy 51 days ago
Just to clarify, it's not the prompt voodoo that caused the affinity for goblins. It's the reward. They rewarded it for mentioning goblins when set to Nerdy, and it's still the same model as the other personalities, so the effects can carry over.
link
airstrike 51 days ago
Makes sense, but I don't know why they'd let said prompt voodoo touch RL. I'm OK with prompting to get the model to, I don't know, write better Rust or build Excel spreadsheets. I am less OK with making it "quirky" or having some "personality" in a way that becomes ingrained in the model for everyone else

TL;DR the cringe nerdy shit should be (optionally) switched on at inference, not as part of RL

link
nearbuy 50 days ago
They do it because training different personalities is more effective than just changing the system prompt. Ever try asking ChatGPT to adopt a specific personality in a prompt? Its standard style bleeds through.

As the article says, the personalities weren't supposed to affect other personalities. OpenAI was as surprised by the goblins as you are. Training can be tricky.

link
surgical_fire 52 days ago
I configured it to use the nerdy personality when I used it to help me on a personal project (setting up a home server, nothing too fancy). LLMs are great at parsing documentation and combing through forums to find out the configurations that matched my goals.

The first time it said something along the lines of "let's use these options to avoid future gremlins haunting you", I sort of rolled my eyes but it was okay, I thought its attempt to sound endearing almost cute. A bit of a "hello fellow kids" attempt at sounding nerdy.

It quickly became noise though. It was extremely overused. Sometimes multiple mentions to goblins in the same reply.

I don't really have an opinion about it, but I sort of came to prefer a more neutral tone instead.

link