I think the lack of control is undesirable. I agree the feature itself can be very useful. The title of this post sounds way more sinister than it appears.
Couldn't I apply the same argument for while google needs to send your call history to its servers to perform number lookup? Yet people cry fowl (rightfully so) about privacy implications.
And remember Apple is a positioning iCloud as a cloud for dummies solution so adding choice comes with trade offs.