Logically the next step is to scan for any copyrighted content and notify authorities that you're watching a movie without paying for it. After all, it's all about catching criminals, how could you possibly object.
That's a fairly large step, though. Apple cares first and foremost about their reputation. If this feature catches a real predator, it is 100% good PR. Every single false positive that makes it into the news is a huge loss, which strongly incentivizes them to avoid that. The last thing I expect them to do is expand the risk surface for something as trivial as copyright enforcement.
> Every single false positive that makes it into the news is a huge loss, which strongly incentivizes them to avoid that.
Just to be clear, "false positive" in this case means an innocent person is accused of trafficking in child sexual abuse material. It's likely they will be raided.
Sure, that's bad if you're Apple, but it's a lot worse if you're the alleged predator.
The thing (many) people are worried about is that, having instituted a backdoor on the user's device, Apple may lose the ability to control how it is used. In the past, their strongest defense when the FBI asked them to use a backdoor to decrypt a user's device password has been "we don't have a backdoor that allows us to decrypt a user's device password, so we couldn't do that for you even if we wanted to".
From now on, when asked to check whether a user's encrypted phone contains arbitrary content the FBI wants to know about, Apple can no longer say "we don't have a way to do that." Sooner or later, you can bet they will start doing it, whether they want to or not.
CASM is already a pretty huge step into legal gray areas. There is no guarantee by anyone that images in the database are illegal. If the database could be guaranteed to only include real photos directly showing sex with a minor I personally would be all for device side scanning even without upload. However who knows what is considered CASM. I would not be at all surprised if it included Guess ads and cartoon parody. Heck in Australia it is illegal to share pictures of adult topless women with small busts because people may pretend it is child porn.
If this feature leads to anyone losing their job due to incorrect criminal accusations it will not even make the papers because we expect the accused are guilty anyway. Apple won't shed a tear until there is a class action.