Actually, banks are for aggregating capital. There is no institution in the Western financial system where you can just store money. The best you can do is put physical cash in a safe deposit box.
AFAICT, it actually isn't in the general case (it may be when done with certain intent or effect), but most US banks now have a blanket prohibition on foreign or US currency in the agreement they require when you rent a safe deposit box.