The App Store run by a central authority with complete control over what can even be available and the ability to modify the delivery at their own whim is certainly a big issue in terms of trusting the integrity of the apps running on a device.
If you suspect yourself to be a selected recipient (e.g. you're Edward Snowden) I reckon you should compile your own binaries. Or read 'Reflections on Trusting Trust'.
Forget the software, the firmware running on the baseband processor can read system memory and send it over the network without you knowing. But that takes lots of effort to target a specific person.
So what do you do? It comes back to making sure that 'they' can only hack some of the people all the time, and all of the people some of the time. It's preventing them hacking all the people all the time I worry about.