|
|
|
|
|
|
by spijdar
9 days ago
|
|
|
Sure they can. I was able to figure out Gemini 2.5 Pro's "Memory" feature's hidden system prompt because the reasoning tokens references the markdown headers by name as "blah blah says I can't refer to this", while the output would never mention them. Yeah, I get that you can jailbreak and get that info anyway. Also that this is specific to front ends like web chat and less about API usage. But as a sibling points out it's also a good way to make post training other models harder. Mostly a "win/win" for the provider. |
|
|