Sure, but couldn't a single cabinet member advise the Queen to withhold assent? As long as they made a compelling argument, the Queen could do so. I'm not claiming this is likely or desirable, only possible in the most extreme situation.
It's generally thought she can exercise her reserve powers precisely once, before they (or the monarchy) are removed for good. Personally ordering the armed forces not to take some antidemocratic action demanded by her ministers in a time of crisis is the most likely scenario, I think.
As with much else in the constitution, the 'on the advice of her ministers' is normally (binding) convention rather than statute.