Not necessarily, if they hack into one system then getting into another isn't automatic. If the passwords are in a separate filesystem/database than the key, and linked only with software, then unless it's the software that's comprised it would still increase the difficulty of getting both the password and key significantly. It also prevents trivial browsing of passwords via sql commands by rouge employees.
With symmetric encryption, probably (assuming the data wasn't gleaned with a purely SQL injection attack). With public key/private key encryption you could probably do it more securely by not letting the private key anywhere near the main app/web servers.
Of course, the more separation you have between the public and private keys, the less convenient it is to actually do anything useful with the plaintext.