[Matumio]

Asking a token-based LM to generate only tokens that contain the letter "Z" is... well.

Like asking someone who prepared a speech to smoothly skip all words with "ei" in them. The LM would basically need to have memorized the set of letters that each of its token stands for, from the training data. I'm surprised it works at all.

[pazimzadeh]

It would be nice if it would tell you when you are asking it something that it is not designed to answer

[Veen]

It doesn't know what it can and can't do. That's why you can never believe what it says about its own capabilities.