I pick Amazon most of the time simply because I can read Kindle books on more devices. This isn't because of any good thing Amazon has done but rather its successful pursuit of snticompeititive policies:
1. Kindles won't read non kindle ebooks. (The kindle fire might if you know how, but it's not easy)
2. iPad will read all competiting ebooks.
3. Amazon's policies coerce publishers to keep prices low, in particular generating a revenue poverty trap for publishing books at prices between $10 and $20.
Actually the practical format you want is MOBI, which Calibre will also convert to. It's a little annoying that everyone else has settled on EPUB, and Kindle doesn't read EPUB. But Calibre will seamlessly convert EPUBs to MOBIs and MOBIs to EPUBs, so the end result is all non-DRMed books can be read on all devices with almost no effort.
Given the ability for different resellers to compete on price, which Apple's anti-competitive rules have effectively taken away, we might not be stuck with a choice between AAPL and AMZN as a source for purchasing "ebooks".