I have not been able to figure out if the YouTube downvote means "this video is bad and should not exist" or "it's a good video but I'm not interested".
Nor has anyone else, which makes attempting to use these mechanisms as "signal" somewhat circumspect, and only gets worse once you take into consideration the mathematical issues with trying to then "average" a bunch of unrelated metrics (something people love to try to do with five star ratings even though it is well-documented as returning garbage; the only reason it is sometimes better than nothing is because nothing is a pretty low bar for a competitor to defeat ;P), but I'd say one of the key issues is actually the sampling bias of "who bothers to vote": different topic posts attract different kinds of people (or are barraged by niche audiences) who are quite likely to react to really awkward things that are ancillary to the correctness or quality.
This problem even happens in places you might fail to notice if you aren't paying attention: as an example, with ratings for hotels, depending on the location--even for what is essentially a cookie-cutter franchise--hotels attract different audiences, and so the ratings people leave mean something different! I noticed this often as a I (used to... damn pandemic ;P) travel a lot, and in some cities (like San Jose) all the reviews of a Marriott would be talking about how useful the rooms are for parties, as a lot of the people booking hotels are doing so for tech companies, whereas in another city (I can't think of an example but imagine some place you would usually go on for a vacation instead of a conference) all the reviews would be focused on whether or not the pool was a fun and safe place for their children. I frankly care about neither of these qualities of a hotel :/.
What you probably want to do is enough "collaborative filtering" to only care about voters who are similar to you. This is the kind of thing people were talking about a lot back in the days of the Netflix prize competition (which had an amazing forum they deleted :/), with k-means clustering of users trying to figure out which subset of votes "matters" to you, and people thought this would be the future and how every site worked... but then Netflix seems to have just gone in a direction of extremely simplistic category affinity (and even removed five star ratings recently). In my case, this would eventually (hopefully) case me to ben heavily weighting people who tend to rate a hotel based on the quality of its bed and the smell of its air, to the exclusion of almost every other rating axis.
The reality is that--for better or for worse--platforms refuse to actually provide rich signaling for these mechanisms (and what they do do is sometimes misunderstood: like, it actually irks me that this grandparent post thinks a Facebook sad react is supposed to be bad somehow: if you say something and I react "Sad" I might very well be commiserating with you... and Facebook knows this so Sad is worth two Likes! fwiw, they did finally decide to make Angry worth 0 ;P) and even--such as the case of Netflix--tend to remove and "simplify" signaling mechanisms over time, and so none of these mechanisms they have "matter" really as they are so low signal and are used so naively.
The only content platform I can think of that "got big" (but isn't anymore) which really tried hard to work on this problem is Slashdot, which asked you why you were downvoting something, and then 1) required you to choose whether you got to comment anywhere at all on the post's thread OR vote on any of the comments and 2) would show your voting decisions to other people who would "meta-moderate" your decisions to decide how good you were at it and then change how often you were given the option of voting. This way you could actually separate out "tyranny of a majority that I think is dumb" from "this comment is actually off-topic or an attempt to incite chaos".