[charcircuit]

This is like saying coding is a known and solved problem and benchmarks for coding is pointless for a general model. And that using a model for coding is an abuse of the multiturn chats AIs are tuned for.

1. The user should be able to prompt the AI to act differently from its default behavior. A human assistant is capable of role playing without always sounding like an assistant.

2. If the user asks the AI to follow the script and not allow unrealistic things to happen it should push back. The user is not always absolutely correct.

[orbital-decay]

You have a point, this might help test the resistance to jailbreaks for example. There are probably better tests for that than OOC roleplaying, I think. At the same time, what you expect is likely not going to happen due to many reasons (e.g. can't optimize for two contradicting objectives), and harness design offers a practical workaround that is being used for years already and is reliable unless the user is actively trying to jailbreak the model+harness.