Anecdotally, I feel that it's mostly because of stereotypes that nerds are socially awkward. In American culture, there's a lot of emphasis on being popular.
Yes, just looking at the replies here, it seems that everyone wants to shoehorn themselves and other into categories. I am a nerd in many ways, but I am also plenty of other things.