I'm not getting your comment. The payload is not encrypted. I think you refer to the signature. The payload can always be decoded. It's just JSON into base64.
Ah, sorry, that was what I was referring to when I said "Presumably you weren't using it to sign the tokens, if they were surprised the client could access them?". I classed that as too obvious for it to be what you meant.