https://en.wikipedia.org/wiki/United_States_Post_Office_Depa...
There is Constitutional authority empowering Congress "To establish post offices and post roads" (https://en.wikipedia.org/wiki/Postal_Clause)
We also used to have a limited form of postal banking here:
https://en.wikipedia.org/wiki/United_States_Postal_Savings_S...