The assumption of woman as victim when there's no indication that she sees it that way is what's degrading. Can you imagine if anytime you were talking to a member of your preferred sex, somebody came running up and yelled to the person you were talking to, "Hey, back off, buddy"?
There's a difference between talking and awkward, unwanted, sexist advances. If you can't tell the difference, then you're right, you probably shouldn't tell anyone off. But just because the victim doesn't say something doesn't mean they don't want something to be said.
Sure, if a guy walks up to a chick and sticks his hand down her pants, that's one thing, because he's clearly out of control. But the example given in the OP was something like "Would you like to have a drink?" I don't think I would be inclined to come charging to the rescue if I saw that. I don't like that women have to deal with unwelcome advances, but I wouldn't feel right rejecting him for her.