[kijin]

Isn't this somewhat compensated for by the extra credit Google gives you for using SSL?

I would also assume that Google is smart enough to take the physical location of your server into account when calculating how much penalty to apply in which searches. Sites that load fast in Germany should have higher ranks in searches from Germany.