[dartos]

One thing I disagree with is saying that EMAI is objective.

There's no machine learning model that is truly objective. They're all biased due to their, usually human generated, datasets. It's impossible to account sufficiently for every scenario in a training set, so these models just give an objective veneer to the biases of those that created the dataset.

This phenomena is well documented with predictive models for crime.

Many arrests happen in low-income areas. The data on arrests skew towards those areas. The predictive models are trained on that data. Using that data, police make more arrests in low income areas. Those arrests get added to the data set Rinse and repeat.

Replace police, arrests, and low-income with anything and it's still true

For example: Company Leadership, promotions, race

[TeMPOraL]

One person's well-documented AI bias is another man's good calibration with reality they don't want to accept.

Unintentional feedback loop amplifying the thing being measured is a problem, yes, but it doesn't stem from predictions themselves - it's decisions and actions informed by the predictions that can amplify the problem instead of reducing it.

[dartos]

Yes, the real problem is using data that is effectively created by a model to further “refine” that model. That’s what closes the bias accumulation loop.

But those not in the know can and do assume that it’s a computer program, so it can’t be biased, which is not the case for predictive models.

[0xcafefood]

It sounds like you're claiming that literally all predictive models just take some initial sampling bias and amplify it over time. Am I reading this right?

[Jabihjo]

Not OP, but I would say yes. And I would argue that humans behave similarly, except that we have an innate sense to question the status quo. For some of us, this trait is more prominent, whereas somebody who is more comfortable with the status quo will usually be stamped a conservative.

[dartos]

Maybe not every single predictive model (I don’t know every single one), but many of them do, yes.

Especially if you use data created as a direct product of that predictive model to further tune that model.

That bias would accumulate.