Use one of those proxy buttons which only load the actual Facebook button when the user clicks/hovers. Unfortunately I can't find it right now, but I think it was posted here on HN a while ago.
EDIT: Here is a Slashdot discussion[1] about a German newspaper who did that, and here's[2] the link to their jQuery plugin who does just that: only load FB code when the user clicks on the button. It's in German, but translation services are good enough nowadays.