Folks, please use your words. Some of us are just trying to make it through Friday without having to login and use show-dead to see both sides of a discussion.
I've sometimes thought that downvotes shouldn't count for much when given by someone who doesn't reply. Count them, just not with nearly as much weight as someone who is willing to explain their disagreement. We already have flagging for marking something is inappropriate.
Visually though, it's not comparable. A reply (often long) is visible but requires reading to garner the sentiment. A grayed out comment is immediately recognizable as being "bad" and "disliked," hence why it's more powerful, and why people do it. It also requires much less effort than typing out why you disagree.