[ftxbro]

Will it still be all like "As an AI language model I cannot ..." or can this fix it? I mean asking to sexy roleplay as Yoda isn't the same level as asking how to discreetly manufacture methamphetamine at industrial scale there are levels people

[Der_Einzige]

No, and in fact I mention that the opposite is the case in the paper I released about constrained text generation: https://paperswithcode.com/paper/most-language-models-can-be...

If you ask ChatGPT to generate personal info, say Social Security numbers, it tells you "sorry hal I can't do that". If you constrain it's vocabulary to only allow numbers and hyphens, well, it absolutely will generate things that look like social security numbers, in spite of the instruction tuning.

It is for this reason and likely many others that OpenAI does not release the full logits