[true_religion]

I think these models have been trained to not accept 'new facts', so they don't take in user input (or the far more problematic search engine, untrusted tool input) and have that change their world view.

However, that doesn't apply when they are told to roleplay a scenario, so its easier to get it to accept and create output with the idea that this true fact you've seen is part of a fictional scenario, than for it to output the same words within the context of the fact being real.

As an aside, I don't that I have to personify AI in explanations and that all discussions revolve around anecdotes, but I only know enough about the maths behind it to be dangerous, not useful. Does anyone else feel this way?