[icegreentea2]

I think the generalized take away from this article, and the position held by the authors is: "Overall, we are not necessarily against this shift to utilitarian logic, but we think it should only be adopted if it is the result of a democratic process, not just because it’s more convenient." and "Public input about specific systems, such as the one we’ve discussed, is not a replacement for broad societal consensus on the underlying moral frameworks.".

I wonder how exactly this would work. As the article identifies, health care in particular is continuously barraged with questions of how to allocate limited resources. I think the article is right to say that the public was probably in the dark to the specifics of this algorithm, and that the transition to utilitarian based decision making frameworks (ie algorithms) was probably -not- arrived by at by a democratic process.

But I think had you run a democratic process on the principle of using utilitarian logic in health care decision making, you would end up with consensus to go ahead. And then this returns us to this specific algorithmic failure. What is the scaleable process to retaining democratic oversight to these algorithms? How far down do we push? ER rooms have triage procedures. Are these in scope? If so, what do the authors imagine the oversight and control process to look like.

[loeg]

Hm, I think the bigger issue presented is that the algorithm in question is heavily biased against younger patients -- it deviates significantly from an ideal utilitarian model.

[icegreentea2]

Right, so there was a flawed implementation. Even if you had democratic consent to "implement a utilitarian organ matching mode", that would not prevent this failure mode.

So what is the governance and oversight framework for ensuring democratic consent from ideation to implementation to monitoring, and how does it differ from what the UK did? The article points out that there were multiple reviews of the algorithm that identified this bias all the way back in 2019. What is the process that connects that feedback with the democratic process to ensure that flawed implementations never deploy, or are adjusted quickly.