Yes, it is frustrating that rather than make an argument, people downvote thoughtful comments as if it was a low quality post. Perhaps the downvoting feature is actually part of the problem.
Frankly, my subjective observation was that downvoting without responding just for disagreeing with a comment increased significantly at the start of the first Trump term.