Because someone can do a man-in-the-middle attack and intercept the right hash and replace it with another one. And how do you verify that the public key is trusted for the first time?
GPG has no central CAs, but relies on a "web of trust" situation. In reality, there's no one central that everyone trusts, so unless the keys are signed by some individual you personally trust, you're down to being reliant on getting valid keys.