It would be nice if it looked at the values of the probabilities and said "I don't understand the question" if the numbers are too low. Or for fun, it could point out how stupid the question was.
Yes, this is an important challenge. There has been a lot of interest in the NLP community right now, particularly around QA tasks [1] Standard supervised models do it well, but zero-shot models still have trouble.
It would be nice, but it's hard to know what probability is "too low". In short, the probability assigned by a model to a sequence of tokens can be arbitrarily low. There are things that are very unlikely to be said, but not impossible... and we still want them to be assignad some non-zero probability by a language model. So it's very difficult to choose a threshold that won't possibly exclude a large part of the sequences recognised by a language model.
1. https://arxiv.org/abs/1806.03822