[duvenaud]

I agree, and this makes my proposed patch a weak solution. I was imagining that the specialness of the token would be reinforced during fine-tuning, but even that wouldn't provide any sort of guarantee.