[throwaway27448]

Demonstrating a company is using this model in court seems non-trivial, no? Or am I interpreting this post incorrectly.

[amazingamazing]

Not at all. It is easy to bake in a specific response to a given prompt in the model.

[throwaway27448]

It is trivial to contrive a distinct model that behaves the same, no?

Regardless, most uses of a model won't provide direct interaction. This entire line of thought seems starved of use. Unless we're talking about LLM providers themselves... but that is a separate topic entirely

[amazingamazing]

no, because you don't know what the "secret prompt" is. there could be hundreds such prompts

[throwaway27448]

Ok, I'll take this at face value... why would someone share this seemingly secret prompt with the prosecution?