You do this by explicitly not having a CLA and by attributing the underlying copyright to the collective authors. Then even a supermajority is effectively unable to relicense.
This only really works if the contributions were made under a copyleft license like GPL. With MIT, it's perfectly allowed to rugpull like this so long as you bury the original copyright line/disclaimer/etc somewhere in your app's equivalent of chrome://credits.
No, with MIT, you are only releasing your subsequent modifications/derivative works under a new license. You can't retroactively change the license to anything that is already MIT.
This doesn’t help if almost all of the contributions come from a single corporation, and now come with a different license attached. As forks prove, license changes typically affect future contributions, not previous ones.
I'm really hoping this string of "open source project goes proprietary" news stories are helping people see the value of licenses like the GPL, which do prevent you from releasing future contributions under a different license unless you own the copyright to 100% of the original code.
Indeed: if remaining open is valued, people should be looking for licenses that prevent it, not ownership by a foundation or similar. That realistically means the GPL.
Unfortunately that cuts to the root cause of the problem, which is not valuing freedom as in speech, but instead only freedom as in beer (or, in the case of a lot of software, free as in mattress).