Also I think getting downvoted for something when you honestly can't see that you did anything wrong is kinda hurtful, which happens mostly when downvoting is used as shorthand for disagreement. I've seen a single comment of mine go up-down-up-down with every refresh which still ended up at +4 or so. Which makes me curious if separate up- and down-counters might offer some additional insight over a single counter.
Yeah, when you talk and track people's facial expressions in real time, you can see which particular things they disagree with, so you have useful feedback. A down vote is sort of like someone running up behind you and king hitting you, for something you said in some other group earlier, and maybe you don't know why. It's not always so non-specific...but not being able to respond to a downvote...makes it an incredibly potent tool to control discourse.
I think there might be a potential problem there in that retaliatory downvoting might be used as a weapon to disincetivise criticism, especially when (unavoidably) defensive retaliatory downvote cliques form.
So I think I would be okay with a mandatory anonymous reply as a precondition for downvoting.