[willbes]

A book called Automating Inequality by Virginia Eubanks touches on how for many years now, we've hid unfair policies behind computer complexity. It was really eye opening to me how what seems to be a poorly implemented system is actually working as intended in order to make life difficult for certain people.

[marcinzm]

Inequality has existed long before computers and is a social problem not a technical problem. You don't solve social problem with technical solutions since people are clever enough to simply find the next tool to enforce the social status quo using.

[willbes]

You are 100% right. The book just laid out exactly what you are saying.

Computers just made the enforcement easier and with less opportunities to break out of it - for example, a sympathetic public servant no longer has the power to make exceptions since "the computer won't allow it."

[P5fRxh5kUvp2th]

^ and this isn't just a public service problem, either.

I'm making this number up, but I swear that 90% of the time when I come across a company that's absolutely floundering under the weight of its own poor decisions, it's because those decisions are enforced by software such that people CAN'T work around it.

What ends up happening is doing anything crosses 15 silo's and each of those silo's absolutely enforces it's authority such that no single person is able to bridge the gap between even two of them.

What makes it hard is that there are legitimate reasons why you don't want a single person being able to write code, push it to production, open firewalls, open access to databases, add ACL's, etc. Depending on the industry, the need for controls should exist, and because there's SOME legitimacy there it gets pushed across the threshold past reasonableness to absolute pain for everyone involved.

---

And if I may, that's where the most dangerous solutions lie. When there's a legitimate use of them but they can be abused.

[wingspar]

Not familiar with the book, but hasn’t ‘sympathetic public servant with power to make exceptions” - major source of inequity? Giving/not giving exceptions for reasons of bias or through bribes?

[willbes]

It depends on the intent of the rules being enforced.

An example given was a state wanted to spend less on low income benefits. They setup an overly complex computer system that decides eligibility and requires participants in the program to fill out multi page forms 100% accurately on a regular basis or their benefits are suspended immediately. There was no way to appeal this decision, only to re-apply.

Employees of the state agency no longer had any insight into why a form was rejected, and could not speed up the process to resubmit it because the participant had long term medical issues and needed potentially lifesaving prescriptions filled. As a result, many people who would otherwise be eligible for benefits had them denied.

This lead to a marked decrease in spending on social services and was considered a win by the state.

[P5fRxh5kUvp2th]

My mother is on SSI due to disability and our experience has been that they'll deny but a re-application or appeal will eventually see it accepted.

Anyone going through this rigamarole, that's always my advice. Keep pressing, they'll eventually accept. It sucks, but you have to play the game.

[galdosdi]

Close but no cigar... It replaces one inequality (bias against those the officials at the top don't like, in favor of those they do) with another one (Bias against those the officials at the bottom don't like, in favor of those they do)

In practice it creates more freedom, because users have no choice in their top authorities (I'd have to move to another state or country if I don't like the laws) but often have a lot of choice in bottom level authorities (if the guy at the DMV is mean I can go to a different location)

I'm not saying it's strictly good, but it's much more complex. IF you are someone who truly believes the top level policy is good and perfect and has no problems, then yes, any kind of bottom level control.

But if your are someone who is harmed by the top level policies, you now have an escape hatch by which a public servant can help you overcome this.

For example once in NY I tried to register a car I had bought out of state and one DMV worker told me I could not because the previous owner was named Johnathan Smith but had signed his name as John Smith. Ridiculous, but it's possible they were following the letter of the law and an automated system would have said the same.

I simply went to a different DMV where a different worker took a more favorable interpretation. If it had been a piece of software making this comparison I would have been screwed.

[P5fRxh5kUvp2th]

Funny that I have another anecdote, also for the DMV.

I owned a vehicle for 15 years and it broke down on me, as a result I dropped insurance on it as I wasn't sure if I wanted to keep it or not.

fast forward 3 months and it turned out to be a relatively cheap repair so I went to get it tagged (it had expired while uninsured) and the woman at the DMV absolutely hammered me on the paperwork I had to show her to the point that even the agent at the insurance company was confused about what she wanted.

I go to another location and lo-and-behold it's no longer a problem. Maybe she was in a bad mood, maybe she didn't like the way I looked. I have no idea, but the point you're making is absolutely relevant in day to day life.

And if I can pontificate slightly further, inequities exist in life, there is no way to completely remove them. Giving autonomy to low level people and squashing the ones that abuse that autonomy is absolutely better for people overall, even if it isn't perfect.

[mynameisvlad]

If someone (a sympathetic public servant) is doi g something to fight inequity, they would not be a "major source of inequity" themselves, no.

That isn't to say that corrupt public servants don't exist, and that they don't look nearly identical if you were to take a brief glance of their actions without looking at their reasoning, but that doesn't mean they're the same people or causing the same end effect.

[stavros]

I think the problem in this comment, and the mistake many others make, is that they consider their viewpoint canonical, and everyone else's viewpoint wrong.

"Sympathetic" and "source of inequity" are subjective. Letting someone get extra points on their test because of their race is equitable to some, unequitable to others. Who decides?

[aidenn0]

There's an argument to be made that differing opinions of equity spread across many dozens of public servants might be a better approximation of true equity than a single opaque model embedded into a computer and applied to all.

[mynameisvlad]

Equity is not the same as equality and was never intended as a way to gauge equality.

Equity, by its very definition, is unequal. Every single thing done in the name of equity is unequal. On purpose. Because the "job" of equity is to right inequalities in the system.

[bmitc]

That doesn’t address the issue. Computers execute stored human knowledge. If that knowledge contains bias, then computers can absolutely propagate that and amplify it.

So yea, inequality is a social problem, but it is amplified by technical problems. We’ve seen this time and time again, the Internet at large being the most glaring example.

We shouldn’t ignore this amplification effect while we wait to solve the social problems. Furthermore, computers are often viewed as some unbiased decision machine, which makes the problems worse.

[robertlagrant]

> You don't solve social problem with technical solutions

Well, you can solve some things. Internet and zoom went some way to solving "how do we work in lockdowns?"

[nradov]

The Internet and Zoom are good things in general. But there was never any valid need for lockdowns at all, and it should be clear to everyone now that lockdowns caused more problems than they solved. So, I don't understand your point.

https://www.cureus.com/articles/107920-increased-incidence-o...

The real social problem we ought to be solving is stripping bureaucrats and politicians of their ability to impose lockdowns on us at all. And there is no technical solution for that.

[robertlagrant]

> But there was never any valid need for lockdowns at all, and it should be clear to everyone now that lockdowns caused more problems than they solved. So, I don't understand your point.

That's not logical. There's no valid need for traffic, but I appreciate Google Maps traffic reports.

[willbes]

One could contend that perhaps the greater social problem solved by Zoom and the Internet was "How do we allow people to have more lucrative jobs without them paying xx% of their salary to live within a commutable area of an in-demand city"?

[robertlagrant]

Well, possibly :) That will drive salaries down a bit, by the same forces that drove them up, but certainly enables massive quality of life increases.

[jamincan]

I think part of the issue is that AI can potentially mask the problem.