[joering2]

From gmail they also have keys to yor website SSL, as majority of SSL cert providers will send those to admin email (form of verification that someone access admin/webmaster account) on file as form of validation (altho there are other options like CNAME in some providers)

[jlgaddis]

They might have a copy of the signed certificate that the CA sent you via e-mail, but they certainly should not have a copy of the private key that goes along with it (unless you did something stupid like let someone else generate the private key for you).