I’ve been thinking about this, too. I have a few caps due to me grinding my teeth for so many years, so for me, the choice is between plastic from a retainer and little bits of whatever the caps are made of.
Older composite tooth fillings and sealants etc do contain BPA, which is an endocrine disruptor that you probably don't want to be grinding down. Recent filling materials have moved to BPA-free alternatives, though. If your caps (crowns?) are made of composite resin, then I'd be concerned if you're grinding them. Many crowns are made of porcelain now days, though.