[pawn]

I think this has huge potential for abuse. Let's say politifact or snopes or both happen to be biased. Let's say they both lean left or both lean right. Now an entire side of the aisle will always be presented by Google as false. I know that's how most people perceive it anyway, but how's it going to look for Google when they're taking a side? Also, I have to wonder whether this will flag things as false until one of those other sites confirms it, or does it default to neutral?

[lallysingh]

These are objective fact checks, aren't they? It's harder to quietly include bias than in say, a newspaper or cable news.

If one/both of them starts getting power-mad with their influence, they can get booted off the list, and replaced, or have others come in next to them.

If two of them say "This is true", and a third says "This is false", then that third one, if it can present good data for it's POV, can quickly become more trusted.

[rjeli]

Except politifact is blatantly biased:

http://www.politifact.com/truth-o-meter/statements/2015/jul/...

http://www.politifact.com/virginia/statements/2016/jun/20/do...

[vinay427]

Not only was the number that Sanders quoted significantly lower, but his campaign provided Politifact with the study that they referenced while Trump's campaign did not respond.

[ruleabidinguser]

The campaign providing them with a study has no effect on how true the statement is, which, as a fact checking organization, was their job to discover. This is surely evidence of, if not bias, gross incompetence.

[eanzenberg]

That's not the point. The point is 2 sides of the aisle reported similar metrics and politifact 'ruled' at the 2 different extremes.

[vinay427]

1. 51% and 59% aren't "similar" on such a large scale.

2. The vagueness of Trump's statement allows for far more misinterpretation, which it is his responsibility to avoid in order to be factual. Sanders mentions a specific age range of 17 to 20, which the study that his campaign later referenced was looking at.

3. As you may know, these "rulings" were clearly not at the two different extremes. Source: http://www.politifact.com/about/

[jnordwick]

If you read the two pages, it is pretty bad. The fact checkers point out the same BLS numbers for each, but then go on to expand into areas Sanders was actually less clear on. They gave him a huge benefit of the doubt to using the think tank numbers, but then they don't apply those same numbers to Trump.

[srssays]

They are not only similar but indiscernible. There is no single definition of "unemployment", therefore there is no single unemployment percentage.

[dragonwriter]

"Similar" is a weasel word; they were notably different numbers, and one had an identified basis which allowed showed a clear connection to the truth despite inaccurate terminology which resulted in it getting not rated True.

And the rulings were not at the two extremes, they were two points apart on the six-level scale, with one one position removed from the positive extreme and the other two positions removed from the negative extreme.

[lallysingh]

They're both really just summaries of Tara Sinclair's view, aren't they? Not that it makes them more objective for showing someone else's bias, but the interpretation of the metric selection is essentially a summary of her statements in both.

[sqzmac]

I looked into this when it made the rounds and I don't agree that it is evidence of intentional bias on the part of Politifact.

I do believe it is a good case study that exposes the weak points of their approach toward condensing a quote down to a true/false rating. I think they would do much better to decompose a quote into separate claims and fact check those separately.

The Sanders quote that was fact checked can be read as making three separate claims, and Politifact discusses each before lumping them together for the true/false stamp:

  1. The "real unemployment rate for young people" is not the same as the typically accepted rate
  2. unemployment for blacks >> unemployment for Hispanics > unemployment for whites
  3. The "real unemployment rate for young people" is 51%, 36%, and 33% for those groups, respectively

Sanders doesn't explicitly spell out (1), but he at least uses the qualifier "real" instead of simply saying "the unemployment rate", and his campaign later clarified what he meant. (2) is supported by the study his campaign supplied. The numbers in (3) match the study his campaign supplied.

In contrast, the Trump quote is: "If you look at what’s going on in this country, African-American youth is an example: 59 percent unemployment rate; 59 percent"

and Politifact is only fact checking the "59 percent unemployment rate for black youths". Trump's campaign did not respond, but the Politifact research found that "it appears likely it comes from a computation of all 16- to 24-year-old blacks who aren’t working and may not even want a job, including high school and college students." Since the fact-check is focused on the statistic itself, this tips it into the false category.

Of course, a potential criticism is that Politifact should be fact-checking the higher-level claim that Trump is making. But from this quote, it's very difficult to tell exactly what this claim is. Trump tends to use words to paint emotional pictures that are intended to resonate with a broad audience, rather than making a chain of argument to support specific claims. How do you fact-check a claim about "what's going on in this country"? Any interpretation of such vague phrasing is going to rely as much on the personal experience of the audience as it will on the actual quote.

So I do agree that this exposes a weakness of Politifact. But in this case the Sanders and Trump claims are not equivalent, and Politifact has a clear chain of reasoning behind their ratings.

Another, higher-level potential criticism is that Politifact, as an organization with limited resources, has to make editorial decisions and clearly can't fact-check every public statement made by a candidate. I don't know how they decide which quotes to fact-check, so I don't rely on them to provide a complete account of what is happening in the world. But I'm happy they're able to dig in deeply in some cases where clarification is needed.

The lesson to take away is to read articles from fact-checking organizations for what they're good at. Read and understand their cited evidence and reasoning, not just the true/false stamp.

[colordrops]

The problem is that often these "fact checking" organisations are posting long subjective arguments for whether something is "true" or not. That is not an objective fact - that is an opinion piece. As long as they are citing verifiable sources for atomic black and white facts then it's OK, but that is often not the case.

[TheAdamAndChe]

There can also be tons of bias in how something is presented. How will google handle a post that has factually correct information presented in a biased way?

[Chaebixi]

Also, aren't a lot of "fact checking" organizations actually explicitly partisan advocacy organizations?

edit: I'm not saying all of them, downvoters. I can't be the first one to think a partisan "fact check" could be an effective political tool.

[Karunamon]

I find that usually the people checking one side's arguments are their partisan opposites. It makes for pretty comprehensive coverage given the adversarial nature, but it also leads to stupid shitflinging when they just cant resist inserting their own opinions into it.

Which the other side checks, and does the same thing.

Basically, politics is an ouroboros.

[lukejduncan]

for snopes and politifact the answer is no.

[wavefunction]

Could you provide a few examples of left-leaning bias for Snopes or other 'verifiers'? I am genuinely interested in considering items from a political viewpoint that is likely different than mine.

[wavefunction]

Never let a poster's earnest and polite request stand in the way of a down vote

[jnordwick]

I'm going to go through the list, but I think that often, I'll use Politifact, skew the results. What a Dem might get a neutral or slightly false will get a Repub a pants on fire or complete lie result. Here is one interesting quote (I only understand the econ stuff so that's where I pulled from:

"Trump’s 100-million figure of "out of work" Americans is highly misleading. This number -- in reality, a bit lower at 94 million"

The overall rating on this was completely false. Really? 7.4% off is "highly misleading". The article speaks to a quote Trump made where he did make a mistake in how the unemployment rate is calculate. He did, but his general point was that those not looking for work aren't represented in the unemployment numbers. And tht is a true claim. Trump put it tht they were considered employed, which is clearly wrong.

The article then veers into territory it should tread lightly on where they build their model of who is really out of work but still wants to work. It isn't very good. They aren't good economists, and more importantly at this point it becomes an opinion piece.

Trump's major sin was confusing the different status of recently unemployment people. I would have given him neutral to somewhat false.

There was on of Bernie being off in his numbers too and he god a mostly right rating. I've seen some real disasters where the statement by a Dem was completely unfactual and that was excused because the opinion-based non-factual analysis behind it was deemed to be true. That is no longer fact checking at all.

http://www.politifact.com/truth-o-meter/statements/2017/apr/...

[hueving]

Many facts being disputed can be either true or false depending on interpretation, so the interpretation decides if the person is a liar or not.

For example, "if you like your doctor, you can keep him/her" is true if you include paying for him/her yourself but false if you limit it to what insurance covers.

This is where sites like politifact really get wiggle room to change the narrative. Just interpret the meaning in the most favorable strictness for who you want to call a liar.

[ruleabidinguser]

I honestly can't tell if the people who defend politifact are unscrupulous or genuine. It's infuriating that, no matter what is said, nothing is ever accepted.

[elevensies]

Most political discussion isn't precise enough to produce a yes/no, and often claims are being made about nebulous stuff like the future, morality, the thoughts and intentions of others. Or, for example, one may display a direct quote from someone without mentioning if they are being paid to spread certain views. The quote may be accurate, but presented in a context that makes it prejudicial. It isn't uncommon to read article of pure rumor and innuendo -- so if something is presented as a rumor do you classify it as "true", because it is true that there is a rumor, or false, because the speaker can't verify the rumor? And then if you mark that false, is all speculation marked false?

[elevensies]

For example, I would personally mark any article false if it included any statements on the goals and mindset of Vladimir Putin, unless the author was Vladimir Putin. There are people who probably can't even tell why their wife is upset with them guessing the mindset of a foreign career intelligence officer, politician and tyrant from thousands of miles away.

[danmaz74]

But often you see completely bogus facts going around. It would already be good to get rid of those.

[mtgx]

He took the "best case scenario" there, or the scenario in which it may be most difficult to screw-up the fact-checking.

What I'm more worried about is what happens when CNN, NBC, and WashPost agree on a story narrative that is is actually false.

I mean, how many media entities said Iraq had WMD before the war? Most of them? In that scenario, will Google be able to figure out who is actually telling the truth? And would "reputable" sites such as those have their articles flagged for falseness if Google does come to a different conclusion?

Or will Google only target lesser known sites for flagging because "nobody cares if they disappear from the internet or are flagged the wrong way anyway" (which is actually the pro-censorship argument).

Let's not forget China has used the "We want to stop false rumors" argument since the creation of the Great Firewall. And surprise, surprise - it wasn't used just to stop "false rumors".

[acdha]

> I mean, how many media entities said Iraq had WMD before the war? Most of them?

You're conflating two different categories of error. In the case of Iraq WMD, the claims were attributed to the government agencies making them and most stories noted when experts (e.g. the UN inspectors, French intelligence, etc.) either disagreed or simply pointed out the lack of corroborating evidence. Additionally, all of the credible media published followups and corrections over time.

What we've been seeing a lot more lately are National Enquirer-style complete fabrications where no detail in the story is backed up by a verifiable source and nothing will be retracted, no matter how flagrantly wrong.

Google could stop the latter by flagging sites which don't cite real sources or retract known fabrications. That doesn't mean that they need to be the content police – a site which wants to say policy X is terrible could still run op-eds or simply quote real people (“Sen. X says other party's plan will …”).

[enknamel]

The problem is that BOTH of these "fact checkers" lean hard left. We say them bending the truth (if not out right lying) in their fact checking. One candidate says a fact and it's marked as true, another candidate says the fact and it's false.

[belovedeagle]

Nonono, you see, 51% and 58% are totally different and also Bernie provided sources which makes the exact same fact true whereas Trump didn't which makes it false. Politifact is completely unbiased.

/s. I'm just getting this comment in here before someone puts it in unsarcastically, which invariably happens on HN and reddit.

[jdoliner]

Politifact is blatantly biased. Look at this example:

http://www.politifact.com/truth-o-meter/statements/2017/feb/...

They state in the article that "The numbers check out" but somehow still conclude that the statement is mostly false.

[pawn]

Okay, I didn't want to say it, but I personally think snopes is biased. I'm going to use a few made-up examples because I don't want take the time to hunt down real examples, but I used to see this occasionally and stopped using them.

Myth: Donald Trump said he loves women.

Verdict: Mostly false

Facts: He said he loves women. He's also said mean things about Rosie O'Donnell.

Myth: Bill Clinton said he loves Mexicans.

Verdict: Mostly True

Facts: He said he loves women. Some women are Mexicans

And yes, sometimes it's that absurdly blatant.

[komali2]

>I don't want to take the time to hunt down real examples

That's fine, but I advise nobody to put any credence behind your claim (snopes is biased) until you are willing and able to provide evidence. Until then, it is safer to continue with the general understanding that snopes is not biased.

You can't just toss shit over the wall and hope for the best, man.

[alphapapa]

> Until then, it is safer to continue with the general understanding that snopes is not biased.

Are you serious? You think it's safer to consider them not biased?

That's just bizarre. Snopes is a business. Their purpose is to make money by generating clicks. Why in the world would you consider them some kind of unbiased, altruistic truth-seekers? Because they say so?

That's the most unsafe understanding you could have, because it leaves you completely vulnerable to deception. Place your trust in them and you let them have their way with your understanding of the world. You're like putty in their hands.

> You can't just toss shit over the wall and hope for the best, man.

Well, you're nailing it to the wall and calling it truth, man. At least he realizes what he's doing.

[joelrunyon]

This wasn't a great comment, but there are examples of Snopes being sort of shady.

https://www.forbes.com/sites/kalevleetaru/2016/12/22/the-dai...

[lallysingh]

I read a good half 3/4 of that article and didn't see Snopes coming out shady. It's one thing to check facts on external events, it's another to essentially testify on events on your own life. The article sets up a bunch of ridiculous expectations and then goes on about how they don't fit them.

EDIT: I read the rest, and I hold the same opinion.

[lovehashbrowns]

If you're saying they do that occasionally, and their bias is as obvious as what you claim with your examples, you should be able to find at least one example of Snopes doing something like that. I don't understand the point of making a claim like that but being too lazy to find actual examples of a clear bias. I don't use Snopes enough to know if they're biased or not; I'm just saying.

[catshirt]

can someone fact check this?

[artursapek]

Politifact is definitely not objective and neutral. They lean left and it's pretty obvious.

[ng12]

The problem with objective fact checks: https://twitter.com/nbcnews/status/785299709342654465?lang=e...

[colordrops]

NBC was pretty obviously in Hillary's camp.

[Jach]

Even if it were all objective fact reporting, related facts can be left out that change the picture. Imagine someone reporting that some new supplement is correlated with a 500% increase in some specific deadly disease, and that whatever study that produced that number is bulletproof. Sounds risky! But the disease's base rate, not reported, is only 10^-6, so 500% over that really isn't that much.

Another fun one is if certain facts about overall dimensions of the data are reported but not about the individual dimensions themselves: https://en.wikipedia.org/wiki/Simpson%27s_paradox

Basically my point is even if we suppose these are objective facts, that's not enough to rule out bias determining which facts are used.

[snowpanda]

> These are objective fact checks, aren't they

Um no..., and they've never been.

[013a]

Let's look at one recent (3 days ago) example from Politifact. [1]

> "There was a very large infrastructure bill that was approved during the Obama administration, a trillion dollars. Nobody ever saw anything being built." - President Trump

Politifact's response?

> "Mostly false" >> It "Underplays the law's actual achievements."

There are _so_ many ethical questions and conclusions you can make about this one article.

On Trump's side?

- The bill wasn't entirely about infrastructure, but he implies that the entire thing was. Wait... would you consider what he said an implication? Or a claim? What percentage of the bill's total funding, actions, and verbiage need to be about infrastructure before it becomes an "infrastructure bill"?

- "Nobody saw anything" --> Hyperbole, which means fact checking is incredibly difficult. Do we fact check on his hyperbolic aim (maybe "The bill didn't fund as many infrastructure projects as it should have") or do we fact check the statement?

But on Politifact's side...

- If we fact check the statement as-is, its not "mostly false"; it is false. At least one person saw an infrastructure project. The article even says so.

- Most reasonable people would interpret his statement as hyperbole. Politifact clearly does in giving him a "mostly false" rating. If we fact check the hyperbole, how do you decide what the actual intention of his statement is?

The point is: When we are fact checking every sentence politicians say, even if they're very short summaries of a much deeper issue, there is no such thing as objective fact. Period.

In this example, Trump cannot reasonably be expected to fully talk about every part of the bill Obama passed. Its a long bill. He summarizes his position on it. This is what comes out. There is hyperbole. There is rounding error. There is factual error. Its imperfect. Humans are imperfect.

The flipside of that is also true: Politicians summarize and use hyperbole, but fact-checkers do the same thing. That politifact article takes thousands of words of research, argumentative facets, points against, points for, hundreds upon hundreds of hours of work... and just says "Mostly false." Does that really explain the situation? Really?

Most readers will not read past that single line: "Mostly false". Yup, Trump lied again, bad Trump! Politifact is responsible enough in providing the rest of the data. And, really, we can place some of the blame on lazy readers who don't read the rest of the article. Not politifact's fault.

Yet, think of the corollary between a lazy reader and a machine. A machine cannot read the rest of that article. It cannot understand the intricacies of this debate; not without much stronger AI than we have today. The algorithm sees "mostly false" and thinks "ok lets factor that into the score."

And that's assuming the algorithm even considers this fact-checking website. What makes a fact checking website reliable? Who gets to decide which websites we trust? Is it that they are right 100% of the time? I've already outlined how much room there is for bias just in this single instance; how can we even truly decide what "right" means when the statements we are fact checking aren't scientific papers, with every data and variable accounted for?

This whole thing is dangerous. I hate it. I hate Google for doing it. I hate Facebook for doing it. They have no respect for how much power they wield. They have no respect for the bubble their decision-makers live in. They are children who, almost accidentally, stumbled upon the nuclear football. They flip switches and dials with no responsibility, and if Australia gets nuked up in the process, they'll be there to sell you iodine.

[1] http://www.politifact.com/truth-o-meter/statements/2017/apr/...

[Chaebixi]

> If one/both of them starts getting power-mad with their influence, they can get booted off the list, and replaced, or have others come in next to them.

How will the algorithm know a source is getting power mad?

[lallysingh]

I'd assume that the list is controlled by some people at Google who make their own decisions.

[belovedeagle]

And I'm positive those people are completely unbiased.

[ed_balls]

That depends how it's implemented. If this is a low pass filter that can tell you if the event happen then the abuse will be close to zero.

[kbenson]

> Let's say politifact or snopes or both happen to be biased. Let's say they both lean left or both lean right.

They outline how to get your fact checking into their system. Presumably if your fact checking is relevant and correct, you can be included in the results. At the point where an organization is trying to be included, but is being excluded or marginalized because of opinion or politics and not because of the validity of their assessments, then we can have a discussion about abuse. Until then, let's use what we have.

> Also, I have to wonder whether this will flag things as false until one of those other sites confirms it, or does it default to neutral?

It's extra information added to things that are fact checked from the knowledge graph. I don't think they'll be tagging everything, since the vast majority of searches will have no associated info. Stub fact check data marked as "pending" could possibly be spidered though, and I suspect items associated with that might come up with an "Unknown" or "Pending" value in the fact check expansion area.