[BasHamer]

Data-driven algorithms are discriminating based upon undesirable/illegal vectors; they are utterly amoral in optimizing their solutions. Even if the algorithm does not have access to the "Age" field, then there are plenty of proxies, like what reunion tour you liked. And the same goes for race, gender, sexual identification, religion, etc.

To solve this we either need the training data to have no illegal/undesired discrimination, or we make the system moral. I think the first is impossible, and the second is what we will do sooner or later.

[biztos]

How would you make the system moral?

Let's say "moral" means "won't discriminate based on X" and the same "system" is used by everyone, which of course it wouldn't be.

So do you make up a bunch of fake "people" who are equal in everything except X, and test that it doesn't advantage/disadvantage the X's? Would that even be possible if the "system" is getting its inputs from social media?

Do you do mandate some kind of audit of the system's decisions, and require it to choose on average the same percentage of Xs as... what? As there are Xs in the general population? In the candidate pool?

I'd love for this kind of thing to work but even in an idealized hypothetical version it's hard to see how it could.

I think in tech we've already shown that shame is no barrier to hiring discrimination, and as HR+AI type filtering systems preselect candidates for you it'll be harder and harder for you or the government or the disadvantaged candidates to even know if you're discriminating.

You'll judge the "system" based solely on whether the set of candidates you got achieved the outcome you needed.

[BasHamer]

train it.

Give it examples that we consider moral and examples of what we consider immoral and have it figure it out. The solutions that the algorithms create are less complex than the data that they base the solutions on; so it should be relatively easy for it to model these solutions as data. We would have to train it on what we consider moral and immoral; that would require us to visualize the solutions in a way that a human can make the determination and provide the feedback.

As far as how we get to the solution, that will probably come when there is a liability for discrimination. So lawsuits like the one mentioned. I think that mandating does not work well, it would be more appropriate to make people liable for the decisions made by amoral systems. This liability would create a demand for moral systems.

[jerf]

"Give it examples that we consider moral"

That's a tall order, honestly. There's a lot of things in the current dominant SV philosophy that are fine and dandy and everybody thinks they agree with everybody else about them as long as everyone carefully agrees to not sit down and actually put numbers on the terms in question ("discrimination is bad!" "I agree!"), but when it comes time to write down concrete rules and provide concrete examples ("hiring a woman is 43.2% preferable to hiring a man; hiring an African American is 23.1% preferable to hiring a Chinese person") are going to make people squirm, and everyone involved in such a project is going to do everything in their power to avoid having to deal with the result.

I bet there's a number of people reading this post right now squirming and deeply, deeply tempted to hit that reply button and start haranguing me about those numbers and how dare I even think such things, as you've been trained to find someone to blame for any occurrence of such words and I'm the only apparent candidate. But I have no attachment to the numbers themselves and I pre-emptively acquiesce to any corrections you'd care to make to them, for the sake of argument. I expect a real model would use more complicated functions of more parameters, I just used simple percentages because they fit into text easily. But any algorithm must produce some sort of result that looks like that, and once you get ten people at a table looking at any given concrete instantiation of this "morality", 9.8 of them are not going to agree it's moral.

I cite the handful of articles we've even seen in peer-reviewed science journals, sometimes linked here on HN, which discuss the discriminatory aspects of this or that current ML system, while scrupulously avoiding answering the question of what exactly a "non-discriminatory" system actually is. It's one of those things that once you see it you can't unsee it. (And given that these papers are nominally mathematical papers by nominally "real scientists", if I were a reviewer I'd "no publish" these papers until they fix that oversight, because it isn't actually that useful to point out that an existing mathematical system fails to conform to a currently-not-existing mathematical standard.)

[biztos]

Yeah but -- assuming this could work in theory -- is it actually possible to give it examples if you don't know all its inputs?

For instance what if when scraping social media it discovers that a tendency to post memes with the color green together with frequent mention of cats correlates to better Python skills, but it happens that Elbonians are forbidden by law to mention cats? Would the system even know that's what it found? Would it even be knowable in the end? Would that be an immoral outcome even if the system didn't know about the Elbonian Anti-Cat Law? And wouldn't you have to know about the correlation already in order to give it "moral" and "immoral" examples?

I agree that litigation (the threat of litigation) is going to remain a factor for a long time, but I see this potentially turning into some kind of black-box system where there might be very serious discrimination but it would be impossible to prove.

[edit: corrected for spell-check, and apologies to any Albanians who like cats. :-)]

[s73v3r_]

"How would you make the system moral?"

Don't use it.