Probably varies from bank to bank, but in my experience you have to specifically buy separate insurance if you want the content of your deposit box insured. The big problem from the bank's point of view is that, unlike your bank account, the bank doesn't know what you have in the box and thus has no idea what to insure it for and no way to verify any claim.
More importantly the bank doesn't profit more from something more valuable being in your safety deposit box so it doesn't make economic sense for them to be the one insuring it.