Yes. CC companies are their own silo. Google aggregates data across multiple sources, and you can be certain that you'll start seeing ads related to purchases you've already made in meatspace.
The same argument could be made for an advertiser peeking into your windows at night and watching you in your spare time. Eventually you have to draw the line.