[yummyfajitas]

This is simply not true. Most algorithms can and will correct for biases in their inputs.

See this (somewhat technical) article where I go into explicit (simulations in numpy) levels of detail:

https://www.chrisstucchio.com/blog/2016/alien_intelligences_...

The best analogy I've come up with for the non-technical is that algorithms are like humans trying to draw inferences on octopus society. Some octopi might have bias against some other octopi, but it's the height of octopusthromorphism to to expect a human to reproduce that bias.

[wrsh07]

This is very optimistic. There are well known and documented cases of ml algorithm bias and its cause [1].

And it's not surprising that data itself contains some biases from the humans creating it. Suppose police are asking machine learning where more crime is committed - there will be a feedback loop. Where are they currently making more arrests? If they spend more time there, the bias will be exaggerated.

The op correctly gauges how we should be cautious. Your post, I'm afraid, is misleading at best.

[1] https://www.google.com/amp/s/www.technologyreview.com/s/6017...

[yummyfajitas]

Of course data contains biases. But again, please read the article I linked; algorithms will have a tendency to correct that bias.

The examples in the article you link to are not algorithmic bias at all. They consist of:

1) Humans at Facebook manipulating trending results.

2) Google's keyword algorithm (accurately) reflecting the fact that people with black names are more likely to have arrest records.

Lets distinguish "bias" from "accurately learning things you wish it wouldn't learn" or "accurately learning things you wish weren't true."

None of what I'm saying is remotely controversial. If I told you statistics could detect and correct bias in a mobile phone compass, you'd just think "cool stats bro". Is this article remotely controversial? https://www.chrisstucchio.com/blog/2016/bayesian_calibration...

The specific feedback loop you describe - variable detection probability => variable # of detections - can be directly mitigated. For a non-controversial example drawn from sensor networks (sensors report events with a delayed eraction, the longer you wait the more events you detect), see here: https://www.chrisstucchio.com/blog/2016/delayed_reactions.ht...

(You can find similar examples all over the place. I just link to the ones I wrote because they spring immediately to mind.)

In a compass, a sensor network, adtech or other quant finance, the idea that machine learning can fix biased inputs is not remotely controversial. The concept that statistics suddenly stops working to fix racism is just silly anthropomorphism.

[wrsh07]

Aha - I think I see our miscommunication. When you say bias you mean statistical bias.

Yes, machine learning is able to correct for that kind of bias - 538's polls forecast is a good example of that.

But you don't get to redefine racial bias to be something innocuous. Yes, black names are more likely to have arrest records, but that "fact" is super misleading [1].

Finally, you're talking past me. I'm not saying that statistics is broken. I'm saying that we should be especially mindful of the OPs point when they say this:

> So what’s your data being fried in? These algorithms train on large collections that you know nothing about. Sites like Google operate on a scale hundreds of times bigger than anything in the humanities. Any irregularities in that training data end up infused into in the classifier.

I think the OP author also has a related post about the kind of bias I'm talking about: http://idlewords.com/talks/sase_panel.htm

[1]: http://www.huffingtonpost.com/kim-farbota/black-crime-rates-...

[yummyfajitas]

Without getting into a dispute about the definition of "bias", I'm saying that algorithms can accurately measure reality even if input(x=white, all else equal) != input(x=black, all else equal).

You are saying that algorithms are accurately measuring a reality you wish were different. I don't disagree with this.

The right thing to do is to actually answer unpleasant moral questions like "if blacks are 4x more likely to be dangerous criminals, what should we do about it?" But I guess overloading the word "bias" is a nice substitute for clearly thinking things through.

[eridius]

The problem is you're modeling a biased reality. And accurately modeling a biased reality may in many cases accentuate the bias. Take for example the previously-mentioned case of using an algorithm to determine where to focus your policing efforts. If the data you have says that more arrests are done in a particular part of the city, then you'll want to put more police there, right? But areas where there are more police will tend to see more arrests. So the fact that you're putting more police in an area where you see more arrests is just going to make the bias more extreme, causing even more arrests there. This causes a feedback loop. So you may be accurately modeling reality, but you're modeling a pre-existing bias and making it worse. And who knows why that pre-existing bias was even there? The fact that there were more arrests there may not be because that area actually has more crime committed, it could be due to other factors, such as racial profiling by police, and in that case your algorithm is now accidentally racist because it's perpetuating racial profiling.

[dragonwriter]

The problems are really twofold:

(1) Defining the proper goals, and

(2) Measuring the right things (such as the real goals of interest rather than biased proxies.)

With police deployments, you are assuming the solution (rather than letting your algorithm optimize it) by saying "I want to put more police where more arrests occur". What you really want is probably something more like (the exact goal may be different, of course) "I want to deploy police resources where it will most effectively reduce the incidence of crime, weighted by some assigned measure of severity." Then let your ML algorithm crunch the various measurable factors and produce an optimum deployment to do that.

(But, then again with that goal -- and similar problems exist with many likely real goals -- you run into the other problem, which is measuring the incidence of crime -- measuring crime reports may be the obvious approach, but there's plenty of evidence that lots of factors can bias crime reports, including communities having bad experience with police being less likely to report crimes.)

[yummyfajitas]

I directly addressed this critique two posts up. Why don't you go read that post?

https://news.ycombinator.com/item?id=12627359

[srean]

Are you saying that it can form a good estimate of the conditional probability ? I can believe that if the sampling process preserves the conditional.

Otherwise one would have to make assumptions about (or in other words, model) the corruption process. The bias compensation machinery then has to be deliberate, wont happen on its own.

Some sampling processes do not modify the conditional. In those cases no special machinery would be required.

[yummyfajitas]

tOne approach is to directly model the corruption process. Being the model-based-Bayesian guy I am, this is something I like to do.

But if your model is sufficiently expressive you don't need to explicitly build or model the corruption process. In the example in my linked blog post, test scores might be biased against blacks. But race is also redundantly encoded, so the algorithm has enough information to fix the bias completely by accident.

Fundamentally what I'm saying here is that bias is a statistics problem and has a statistics solution. Insofar as your complaint is algorithms finding the wrong answer, the solution is better stats.

And nothing whatsoever that I've said here would be remotely controversial if the topic were remote sensing.

[mrow84]

To correct biased measurements (in a careful way) you need

1. Enough knowledge about the structure of the bias to be able to devise a model for it.

2. Some measurements from which to fit the model, with errors that are uncorrelated with the errors in your original data.

These things are not always easy to obtain, even in relatively mundane settings. It is also a distinctly non-automatic procedure - it requires someone to decide that a bias exists, to model it, obtain the relevant data, and fit the bias correction model, all before they can begin to obtain unbiased (or probably just less-biased) measurements.

[yummyfajitas]

I'm not making the claim that an algorithm magically fixes everything. I'm claiming that sometimes they do which makes bias less likely to be present in the ML model.

You don't need a human data scientist to decide bias exists, model it and fix it at all. If you read the post I linked to, you can observe a synthetic example of linear regression (with redundant encodings) accidentally fixing bias.

So yes, if your model is expressive enough and you have sufficient data, it will automatically fix bias. Is it really shocking that an algorithm which is good at finding hidden patterns will find a hidden pattern?

[mrow84]

I don't really understand that claim. You are explicitly adding a bias that is linearly dependant on your race variable, and then allowing your regression to recover that bias by introducing noisy measurements of race (which you as the modeller knew was the thing causing the bias). As you say, that is unsurprising.

That result does not, however, address my point, which is that if the structure of the bias is difficult to understand, or perhaps even just difficult to model, and if relevant measurements (with errors that are uncorrelated with your original errors) are unavailable, then bias correction is essentially impossible.

[yummyfajitas]

The point is that the bias is linear, and my model is linear, so the model fixes things. The example is synthetic (so we could know what the right answer is and check if we recover it) so of course I put everything in.

In the linked article, I explicitly reference a real world case where the same linear model was used to discover that grades and test scores are biased in favor of blacks: http://ftp.iza.org/dp8733.pdf

In more complicated situations, the bias would need to be amenable to detection by a neural network, an SVM or random forest. The entire purpose of models like this is that lots of hidden patterns are detected.

Even if relevant measurements are unavailable, one can use redundant encoding to fix bias. Delip Rao explains redundant encoding here, for example, though he is more concerned that ML models might learn facts he wants to remain hidden: http://deliprao.com/archives/129

[mrow84]

To remain with the example in your blog post, your model fixed things because the implicit bias model was correct (linear dependance on race), and the data were available, either directly (via the race variable) in the "What if measurements are biased?" section, or indirectly (via the noisy redundantly-encoded race variables) in the "What if we scrub race, but redundantly encode it?" section.

In the first of those two sections you yourself note how bias correction is not possible without the relevant data: "If we scrubbed the data this result would be impossible. Running least squares on scrubbed data yields alpha = [ 0.29878373, 0.30869833] - we can't correct for bias because we don't know the variable being biased on."

I'm not disputing that bias correction is possible, only that it can be much harder than you seem to be implying, with statements like "Most algorithms can and will correct for biases in their inputs.", and "Of course data contains biases. But again, please read the article I linked; algorithms will have a tendency to correct that bias."

I have some experience with bias correction in (ocean) weather forecasting, and in that domain there were problems both with the difficulty of modelling the bias structure, and with obtaining measurements reliable enough for bias correction.

[yummyfajitas]

What I'm disputing is this:

Machine learning does not have less bias than human researchers. It is simply magnified at scale.

This is fundamentally wrong. Given data on the biasing factor, most algorithms will try to use it and improve things. Sometimes information is unavailable. On net there is a reason why many algorithms will reduce bias, and no particular reason why they would increase it equally in the remaining cases.