[woodrowbarlow]

"a computer can never be held accountable, therefore a computer must never make a [legal] decision"

in my observation: when humans are automated out of a process due to the human element being inconvenient, the perceived efficiency gains are often because wronged individuals have less recourse in the automated system.

[cjs_ac]

There is a widespread belief (on which I am personally undecided and not interested in attempts to sway me one way or the other) in the UK that asylum seekers already have too much access to recourse. Making it easier and cheaper to reject asylum applications within the requirements of international human rights law is probably an implicit aim in this project.

[iso1631]

When you put the computer in charge, it tends to say no

https://www.youtube.com/watch?v=x0YGZPycMEU

[SpicyLemonZest]

It might be helpful to be a bit more specific about what's being automated. One story from the underlying report (https://assets.publishing.service.gov.uk/media/687f6f9dfdc19...):

> In another example, a Vietnamese national was initially given the benefit of the doubt at the first triage that took place in the waiting area. The CIO and social worker commented on his “soft face”, which they said was consistent with his claimed age of 17. However, his “developed shoulders” and “huge hands” cast doubt for them, as did a “tiny bit of stubble” that they noticed when they asked him to raise his chin. The CIO and social worker told inspectors afterwards that Vietnamese young people were typically difficult to assess because they “did not have the same ageing process”, and “did not show signs of ageing”. When asked where the evidence for this was, they said that it was knowledge gained through their own experience. The social worker said, “It is just genetics”, but was unable to support this with evidence.

If I had to choose between being judged by an AI model and being judged based on ad hoc stereotypes of what my race's shoulders and hands typically look like, I'd definitely pick the AI.

[ninalanyon]

Isn't the AI going to use the same ad hoc stereotypes?

[SpicyLemonZest]

I understand that "no" isn't the answer you're looking for but I'm not sure what else to say in response. A computer system is the opposite of an ad hoc stereotype; it can be directly tested for problems and those problems can be corrected if found.

It's a problem when people use this kind of system to circumvent the question of "do we have to make this judgment at all". We shouldn't, for example, predict from someone's photo how likely they are to commit a crime, so we're rightly skeptical of people who try to argue about system X or system Y might better predict it.

But as the source article covers, the UK's asylum laws require it to make this age judgment, because child migrants are entitled to special programs separate from adult migrants on account of their vulnerable status.

[LtWorf]

Are you familiar with how AI gets trained?

[SpicyLemonZest]

I am. I would expect an age classification model to be trained on a dataset containing pictures of people with known ages and their known age.

It's true that the model might develop the same strange belief that large hands prove a Vietnamese person is not 17, if the training data is biased in that way. There's no perfect solution.

[LtWorf]

Yes but those people will be americans or europeans.