[alpaca128]

There are known cases of police faking AI-generated evidence[0]. There's no reason why Apple would be immune against such things. And the recent British post office scandal shows that even without manipulation false faith in technology as evidence can destroy hundreds of lives. The low chance of an error going through that whole chain of checks also increases the trust in that system even in the case of a false positive.

And all this is assuming it will never be expanded from CSAM to other content. Apple is already rolling out a censored version of iOS in China.

[0] https://www.vice.com/en/article/qj8xbq/police-are-telling-sh...