They do offer credit cards through third parties. They also have the Amazon payment service that allows third parties to process transactions through Amazon.
If Amazon could drop $13B on a national grocery retailer they could just as easily drop some number of billions to buy every podunk regional bank willing to sell and merge them into the Bank of Amazonia.
If Elon Musk doesn't do it first to enable easy financing for Elon Musk-fans' buying everything Elon Musk. Musk.
No, they don't really control the money. If all amazon balances were gone it would not be a significant event for the US. Evil corp is like a Bank of America or Wells Fargo. If they close, people can't pay rent, food, etc. For Amazon that's (luckily) not the case (yet).
https://www.geekwire.com/2017/amazon-gives-2-rewards-back-pr...