[nybble41]

You're probably correct that I was thinking of the scrapped (or deferred) iCloud Photos proposal. Though this part:

> Also, it uses on-device machine learning to detect and blur NSFW photos. They even removed the feature that notifies the parents if the child chooses to view a photo that was detected as NSFW anyway, so the contents of messages are not sent to Apple or anyone else.

… suggests that there was something similar in iMessage at one point, even if it was later removed. The "on-device learning" (or rather, on-device classification) means you're effectively sharing the data with Apple's agent running on the device, and the user doesn't have the ability to turn that off. Though it wouldn't be unreasonable to consider the parent who authorized this to be one "end" of the conversation since there is a minor involved—assuming there actually is a minor involved. (These "parental controls" have been abused to monitor adults.) It would be best if that fact was somehow communicated to all the other participants, for example with a "parental control active" or "monitored account" badge on the user's icon & profile.