Hacker News new | ask | show | jobs
by BoorishBears 941 days ago
Everyone will reply that it's impossible, but not leaking the system prompt is pretty easy if you have control over the interface.

Even without resorting to tricks like manual filtering, once the prompt and output format are complex enough, the model struggles to apply attention in a way that results in regurgitating the original prompt.