I've opted out every time I traveled prior to the introduction of (opt-in, non-frequent flyer based) Precheck, so 8-10 times per year from 2010-2014(?) and never had a notable issue.
As the other child comment says, rights atrophy if you don't exercise them.
If you're traveling with someone you know, one trick to ending the pat down early is to try to embarrass the agent while in a sensitive area. My favorite is "you know, he's only doing this because his girlfriend dumped him and is lonely". ~50% of the time they stop immediately and say "you're good".
Embarrassing the TSA agents might work sometimes, but some of them are also known to be vindictive and petty towards anybody who even seems like they might be challenging their authority. Given the possibility, however remote, of accidentally angering a vindictive and petty person who has power over my liberty and/or belongings... I think I'd just shut up and endure the pat-down.
I opt out every time, or did before I got precheck (I know, irony). I never had anything remotely like this experience, and my shaving soap has set off the gas chromatograph machine twice. I still bring it with me when I travel (it doesn't always trigger the machine, but that's a different discussion about the pointlessness of a machine with a lousy false positive rate being used to test for something with a vanishingly small prior).
My secret? I'm an average looking White Male in the US. Nobody ever thinks _even for a second_ that I could be a threat. Last time my soap got me taken to secondary screening, TSA left me alone in the room with their walkie talkie charging station. With all my stuff. Unsupervised.
I'm in the same position. And use the same day bag for train travel as my carry-on, meaning I've brought packs of razor blades on planes by accident more than once. (And a hunting knife once after a camping trip). The theater around keeping us safe is really about protecting us from non-whites
As the other child comment says, rights atrophy if you don't exercise them.