[heavyset_go]

The HF part of RLHF to refine the output of LLMs also happens in these places

[astrange]

Note RLHF can only perform selection on existing model outputs, adding new data is SFT or else just more pretraining.

ChatGPT speaking African English was mostly just 3.5. 4o speaks like a TikTok user from LA. 5 seems kind of generic.