[betterunix]

You left out speed. The reason we do not use PKE exclusively is that we do not want to wait a week to encrypt a megabyte of plaintext. In fact, it is not so much that PKE is what is slow; rather it is that theoretical constructions, which are the only thing we have for PKE but which also exist for symmetric ciphers, hash functions, etc. are slow.

Also, while PKE is sensitive to implementation issues and parameter choices, you can have much higher assurance that there are no theoretical weaknesses than you can have with something like AES. With PKE you usually have a proof that the security of the system depends on the hardness of one problem, regardless of the specific attack strategy the adversary uses. We do not say that ElGamal is secure against chosen plaintext attacks because we ran a battery of tests designed to detect vulnerabilities to particular CPA strategies; we say it is secure because we can prove that any CPA attack on ElGamal can be used to solve the DLOG problem, and so we only really have to worry about the lower bound on solutions to one problem. With something like AES we only test for certain attack strategies and a few general heuristics that suggest a block cipher is secure.

This is not to say that AES is not secure, nor that PKE is a magic bullet.