The idea only works if you keep your private keys. If you use an online Bitcoin “bank”, then you do have to worry about them disappearing. cough Mt Gox cough
You can avoid having to deal with a fiat bank going insolvent by personally managing your money as cash stored under your mattress, too. But there are plenty of reasons you wouldn't want to do that.
Yes. I’m not saying it’s a solution. Just saying that there is a way to avoid the exchanges/“banks” going bankrupt. Reading it over, it does feel like a “technically correct” answer and not a good one.