They are insured in the US, but that's a relatively recent development. Before 1934, when your bank failed, you lost your money, which was why there were runs on banks during financial panics (1929, for example). But it is important to note that there is nothing special about a bank that makes it insured. You could set up a bank that is not FDIC insured. Maybe offer people a higher rate of interest the entice them to come. You can set up a brokerage that is not SIPC insured, too.
If MtGox were managing real money or stocks, and did what they are alleged to have done, I expect there would be some jail time. I doubt it's worth the Japanese government's time to criminally prosecute a few million dollars of new, unregulated currency-equivalent, but if they do prosecute, I'm guessing there is a serious risk of jail time here.
Depending on the nature of the risk, yes, that might be enough insurance.
Bank deposit insurance is fascinating: it largely exists to prevent bank runs, which are caused by people believing they won't get their money out. Having any insurance means that people have less reason to believe there's a chance they won't get paid back, which decreases the probability of a run on the bank, which decreases the risk associated with the insurance, which decreases the amount of insurance needed.
Firstly, your link says nothing about customer deposits, it's about bank derivative holdings. Which aren't insured by FDIC.
But, yes, because it's pretty unlikely that all banks will fail at once, it's not necessary for the FDIC to have on hand a sum matching all insured US bank deposits.
There are also limits on coverage. If you keep $1 million in your savings account, you won't get all of it if the bank goes under.