[mst]

> If we take their past performance as evidence of what is likely to happen moving forward, we will see, just for example a conservatives shut out of the public conversation for saying exactly the same thing a liberal is given a pass on.

There are definitely examples of fact checks of very similar claims being resolved very differently, and the further left person in question being the one given gentler treatment. I'm not sure how you'd effectively test the dataset to determine whether this is just statistical noise, though.

On the other hand - whether you can test it or not, the question at hand seems to be about perception of bias rather than actual bias. I wonder if an adversarial style system wherein, say, both supporters and detractors wrote up their best cases for why something was true/false and then the best arguments were presented as some sort of a ... duel? ... would provide better information. Crowdsource the arguments well enough and the classes go away at that level, and using the resulting information to filter might be rather more useful.

The only things I'm really sure of here, sadly, are that doing nothing doesn't seem to be working particularly well, but every time we attempt to do something we find that doing nothing looks increasingly like at least a local maximum.