| > I do think you missed the point. Please make an effort to engage with the comments I make, rather than making guesses about my mental state. > Me: The human bureaucracy is suppose to be there to determine the quality of the flags and analyze whether there is any discrimination at play. A company that lacks this human element is negligent and should be held responsible. The human bureaucracy doesn’t do that very well. The human bureaucracy is deeply flawed and has limited skills. We can assign blame to the human bureaucracy for its failings all we want, but if we want to effect change then it’s necessary to include a broader range of factors in out fault analysis. In other words, “assigning blame” is a low-stakes political game, and “root-cause analysis” is what really matters. This is like the 737 MAX failures. You can say that it’s the pilot’s responsibility to fly the plane correctly—but the fact is, pilots have a limited amount of skill and focus, and can’t overcome any arbitrary failing of technology. So we rightly attribute the problem to the design of the system, of which the human is only one component. This grading software is like the 737 MAX—it’s software that, as part of a complete system including non-software components like humans, does a bad job and needs repair. The 737 MAX reports listed something like NINE different root causes. I don’t understand this absolutist viewpoint that the human bureaucracy is the ONLY thing that you need to protect you from bad software. There are multiple root causes, and the bad software is one of them. > Hence why the humans should be held responsible for not addressing bias in their system. And why the actions of an algorithm should be the responsibility of its creators. So you’re saying that there’s a problem with the software, and that we shouldn’t place all the blame on the college administrators? Isn’t that what I’m saying? > But if you believe someone has the right to have guns while others don’t, im hard pressed to see any other determining factor except who has the gun. I do believe that not everyone should have the right to own guns, but if you’re interested in arguing with me about it, I won’t engage. If the comparison doesn’t work for you, think of something less emotionally charged like the 737 MAX or the Tesla Autopilot—both are scenarios where we rightly cite the software / automation as a root cause in accidents. |
There are multiple intermediate causes, and all of them are the responsibility of the human bureaucracy—including, to the extent it contributes, the selection, use, and failure to correct bad software—and all of them stem from one root cause, to wit, that the bureaucracy faces insufficient consequences for it's failures and thus lacks motivation to do it's job well.
Now, were the analysis being performed on behalf of the bureaucracy because they had decided to do their job, rather than being part of a discussion outside of them, the causes which are intermediate from a global perspective would be root causes, sure. Context matters.