[wycy]

They do note the random chance bit, and they also note that it's better than humans could judge on their own and even, surprisingly, better than judged by a personality questionnaire.

> Political orientation was correctly classified in 72% of liberal–conservative face pairs, remarkably better than chance (50%), human accuracy (55%), or one afforded by a 100-item personality questionnaire (66%).

[klmadfejno]

Were the humans experts or just random people though?

The real question is whether the tool can beat a lookup table of age, race, and gender probabilities. The tool isn't going to be winning points of phrenology here. Weight, hair color, and hairstyle would also likely tell you a lot.

I don't have any particular reason to believe this tool wouldn't work, but let's not pretend it's getting their by phrenology-esque topologies of people's faces.

A randomly chosen black individual in the united states has a > 72% chance of leaning democrat. A randomly chosen hispanic individual is ~55-65% chance of leaning democrat. I don't find it crazy to imagine they've got a few other smaller features to boost it.

[wycy]

It further notes:

> Accuracy remained high (69%) even when controlling for age, gender, and ethnicity.

[sudosysgen]

Did it? If you control for gender but not sex, you can use the difference to predict ideology. And for ethnicity, there are subethnicities that matter too - white Italian and white German have different proclivities.

[klmadfejno]

How does one calculate a metric like that?

[542354234235]

Simplistically, let’s take the above statistic “A randomly chosen black individual in the united states has a 72% chance of leaning democrat” at face value. So, a coin flip would be lower than 50-50 because someone of that race in that country does not have a 50 50 chance. So you would adjust the chance to 72-28 and compare that to the Facial recognition results. If you find that the results are the same, then you know that the Facial Recognition not picking up on anything beyond race. If the results are different, you know the FR is picking up on something in addition to race.

Really it is more complex than that, but fundamentally you try to say “how accurate can we be using just age, gender, and ethnicity” and use that as your controlled benchmark.

[klmadfejno]

I understand what they're implying by "adjusted accuracy". My point is that I'm not sure that metric really makes sense, because "accuracy" isn't a particularly useful metric to begin with. It depends entirely on the sample distribution. "Always guess not fraud" will be 99.9% "accurate" for most use cases.

I'm asking what the literal metric is.

edit: and I don't think your explanation really works for accuracy, because accuracy isn't a relative measure, like, say, R2.

[simiones]

They explain: they tested predictions on pairs of faces of teh same gender, ethnicity and age. The result was 69% instead of 72% apparently.

[klmadfejno]

Hmm, the actual phrasing is:

> The accuracy is expressed as AUC, or a fraction of correct guesses when distinguishing between all possible pairs of faces—one conservative and one liberal.

I've never seen something like this. Maybe this is a normal procedure?

But I would be worried that the number of old black conservative women would be really small. Seems a bit sketchy

[tylerrobinson]

By performing the analysis within each of those subgroups.

[klmadfejno]

`f(x) = return 'Liberal'` will get you a great accuracy running the analysis within a subset of black women.