[xpe]

There is a strong chance that bias will affect rankings, to some degree, but:

1. Bias is not all or nothing.

2. There is a large degree of commonality about baseline ethics and quality of life. Polarization accentuates wedge issues disproportionately.

3. There are classes of statements which can be factually assessed.

4. Don’t let the perfect be the enemy of the good. A step in the right direction combined with other forces can make a positive difference.

[cheeseomlit]

My objection isn't that its good but not perfect, I don't think itd be good at all. So far everyone who's tried to be a neutral arbiter of truth has been a political hack, like politifact and groups like that. Training the public to trust things just because they got a good score from some agency is not going to improve their ability to think critically and be informed, the false sense of legitimacy (and the ability to apply the rules unevenly to make your enemies look illegitimate) would make the problem even more odious

[mistermann]

What if the agency included a detailed description of the high quality thinking process (assuming they had one, no current sites do) that led to the conclusion, in turn teaching people how to think?

This comment section, in this smarter than usual community, is chock full of lazy, heuristic thinking. We can't even try to not make errors on certain/most subjects, it is our nature, suggesting we do not is socially punished.

[cheeseomlit]

That would help, laying out the process used to reach the conclusion in a clear methodical way would be essential. But in general it feels like a technical solution to a social problem- so much of what goes on in the world isn't clear or methodical, so many events that cant possibly be quantified or verified 100% by a third party- and setting the expectation that they can is IMO placing too much trust in the journalistic process. The reader should always have a seed of doubt, always question the motives of the messenger, and I just don't want that doubt to be assuaged while its still warranted.

This approach may work well for reporting that deals with data and science, things that can be incontrovertibly proven, which counts for a lot. But I think that same approach might fall apart when applied to reporting on complex social and political issues.

Then again, the camera has been a technical solution to the same social problem to some extent. Vietnam was a culture shock for a lot of Americans because they could see real firsthand footage of the events for a change, and that increase in awareness through reporting caused a social reckoning. So maybe its just a question of visibility, seeing is believing

[mistermann]

> But in general it feels like a technical solution to a social problem- so much of what goes on in the world isn't clear or methodical, so many events that cant possibly be quantified or verified 100% by a third party- and setting the expectation that they can is IMO placing too much trust in the journalistic process. The reader should always have a seed of doubt, always question the motives of the messenger, and I just don't want that doubt to be assuaged while its still warranted.

I think it would be a good thing if people were introduced to the concept of the unknown, and learned for the first time in their life to be able to reliably identify it.

And it wouldn't be journalists doing this work, this would have to be an entirely new job role, blending skills from several disciplines (philosophy, psychology, geopolitics, etc).

[xpe]

If a rater has a clear set of evaluation criteria and a moderation log, does that do enough to change your view on the value add?

[cheeseomlit]

Maybe- there are just so many variables, and the trust has been so eroded already. It'd have to be super strict for me to be on board, to the point where it's less journalism and more technical writing. "Just the facts ma'am" to the extreme, every sentence is a statement of verifiable fact- no conjecture or interpretation, all that is left to the reader. You won't get many clicks that way, though.