I see that it has also gotten mainstream news coverage as some kind of lesson about the dangers of machine learning. The real problem is they didn't have data that could answer the question they had, P(Death|No hospitalization), so instead they fit models to answer a different question, P(Death|Hospitalization).
Then they didn't like that the complex models answered the second question too well, so they used simpler ones that made it easier to manually filter out any results that didn't make sense as answers to the first question (which isn't one they could answer to begin with).
No model they fit is safe. You could only use one limited to domains where P(Death|No hospitalization) ~ P(Death|Hospitalization), which isn't something they assessed.
I see that it has also gotten mainstream news coverage as some kind of lesson about the dangers of machine learning. The real problem is they didn't have data that could answer the question they had, P(Death|No hospitalization), so instead they fit models to answer a different question, P(Death|Hospitalization).
Then they didn't like that the complex models answered the second question too well, so they used simpler ones that made it easier to manually filter out any results that didn't make sense as answers to the first question (which isn't one they could answer to begin with).
No model they fit is safe. You could only use one limited to domains where P(Death|No hospitalization) ~ P(Death|Hospitalization), which isn't something they assessed.