| This headline and article are horrible and misrepresent the problem and the outcome. The paper is about the manual process of re-assigning a credit score on a scale of 1 to 15 based on other customer criteria. Really the fact that this process exists at all shows that their initial credit scoring approach is flawed or too simplistic. The argument of "just replace it with an if statement" does not hold up in this scenario. So this is not a "if number big lend. If number small no lend" problem. Its a 15 way multi-class classification problem. They even give a baseline for what happens if they randomly pick or always pick the biggest class in the paper > As is typical in machine learning we also report the Accuracy p-value computed
from a one-sided test (Kuhn et al., 2008) which compares the prediction accuracy to the "no information rate", which
is the largest class percentage in the data (23.85%). So yeah, 95% is somewhat better than 23.85%. I agree with the general sentiment that is is likely a fairly straight forward problem to predict if you are familiar with the bank's operating procedures as there is no way these individuals are making their own risk models and independent decisions. They are there to follow the rules and provide human accountability. An error analysis on the items the model couldn't predict would definitely have been most interesting. |