[embedding-shape]

The latter ("facilitating distillation") is a real concern from the labs I'm sure, but the first part I don't understand what you mean, or you misunderstand that paper, it's about "prompt injection" via manipulating the reasoning, not about the model reasoning by itself and somehow that leading to more prompt injections. They're quite literally maliciously rewriting the reasoning as the model reasons, not just showing it to the end-user, two very different things.

[spijdar]

Yeah, signing would prevent the paper's attack, but seeing the "raw reasoning" is helpful in figuring out what triggers refusals or (especially) what in the system prompt is guiding a specific refusal.

Gemini (the web UI) used to show raw reasoning, or at least a more detailed summary of its reasoning, less than a year ago. Complete with markdown and weird spelling idiosyncracies, so I'd lead towards "real reasoning", but who knows. The reasoning block leaked the system prompt way more often than the response block did, and you could figure out why it would refuse a request through the reasoning, even if the response itself refused to elaborate. This is, presumably, why they stopped showing it. No loss for them, just prevents "pesky users" from low hanging fruit snooping.

(Gemma 4's reasoning and output remind me strongly of what I remember Gemini 2.5/3's reasoning to be, as an aside. I guess that's obvious, but Gemma 3 felt like a totally different model, while 4 feels very Gemini-ish.)

[benob]

What matters for this injection strategy to work is to follow quite closely the style of the reasoning. It's particularly effective if you copy reasoning from the same context. If you cannot see the reasoning, you cannot duplicate it's style.

That said, including instances of the attack in training is already a good countermeasure.