[eyelidlessness]

But that’s the problem. The entire premise of these algorithms is based around what you (or the developers producing it) care about. It’s the common thread between image crops preferring white faces and women’s breasts, and automated cars preferring dead black pedestrians over vehicle collisions.

If you don’t have the capacity to use new technologies without increasing harm, maybe you don’t have the capacity to use them.

[refenestrator]

No, I'm positing the premise that it was shipped with an absolute minimum of QA that didn't approach the level of trying to build 'inclusive' reference sets, on the cheap. It wasn't about caring in terms of priority of what was tested, it was about NOT caring that much in any direction and shipping it.

And it was a naive image cropping algorithm, years ago, and not making use of any sophisticated 'new technologies'. The beauty of the algorithm is that it was a simple function that could have been written in 1975 and required no training, deep learning or any of that. If you want to talk self-driving cars, you've got a much more relevant measure of harm and I'm right there with you.

As it is, I'd say there's a disconnect between where you and those years-ago shoestring developers stand on Maslow's hierarchy of needs. They were being scrappy with limited resources, and you're mad at them for not having an amount of QA that would have seemed unbelievable to them under their resource constraints.

[eyelidlessness]

> No, I'm positing the premise that it was shipped with an absolute minimum of QA that didn't approach the level of trying to build 'inclusive' reference sets, on the cheap. It wasn't about caring in terms of priority of what was tested, it was about NOT caring that much in any direction and shipping it.

Right. I’m saying that not caring is the problem. We mostly agree on the facts. I’m objecting to not caring as an acceptable position.

We’re not talking about shoestring developers. We’re talking about platforms that millions of people including heads of state use to affect billions of people’s lives.

[refenestrator]

> We’re not talking about shoestring developers. We’re talking about platforms that millions of people including heads of state use to affect billions of people’s lives.

Jedberg was >5 years ago, maybe closer to 10 for this algorithm. They were relatively shoestring compared to now, and especially compared to current-day FAANG.

[eyelidlessness]

Well I don't see any sense continuing this discussion, since you just keep refusing to engage with the harm done by doing something without caring about its impact.

[refenestrator]

It's an incredibly privileged definition of 'harm done'. They had less budget and manpower than you wish they had, and their shitty off-brand cropping algo occasionally had too much titty. If that's the level of harm that registers with you, you're clearly doing very well.

[eyelidlessness]

I’m gonna regret responding after saying I was removing myself but I’m hoping this will help you or at least other people reading understand what the harm is.

I’m not objecting to an algorithm that exposes me to more cleavage. I couldn’t care less. I’m objecting to an algorithm that exposes millions of men to a subtle but routine equation of men with faces, and women with breasts. It’s not a privilege for women to expect not to be casually sexualized because someone thought their algorithm was helpful but didn’t bother to find out it wasn’t.

Likewise the impact of racial bias. I’m not offended by seeing white faces. But it’s not a privileged position for POC to object to being basically erased from images because whoever developed the algorithm didn’t bother acknowledging more than one skin tone exists.

These things have real impact, not just on how it affects the people who are directly underrepresented. It also affects how the people who are overrepresented perceive and ultimately act toward them.