[xyzzyz]

that a model could easily mistake race as a causal factor.

Statistical models as used in real world systems don’t have a concept of a “casual factor”. It literally doesn’t matter for the model why in certain zip codes, there is more property crime. It doesn’t care if it’s caused by poverty of residents, by pigmentation of their skin, by lead in the paint, or by the cultural traits of residents. All it cares about is the correlation: if the risk is higher, the insurance premiums go up too. For some it might seem unfair, and for some groups such statistical discrimination might be illegal (though not for all, eg. it’s perfectly legal to charge men higher insurance rates, which suggests that the moral principle here is not equality, but rather compensation for historical mistreatment), but without a doubt, from the model’s and business perspective, such reasoning is undoubtedly correct.