The companies that provide the cash back get way less than 3-5% of the gross purchase price. Merchant fees, at least for Visa, don't even approach 5%[0] -- it's more like 1.5 to 3%. And they're definitely not spending all that fee on rewards (and thus taking no profit).
Another thing to consider is that no payment method is free for the merchant. Even with cash, you need to pay Loomis, or you need to pay an employee to physically visit a bank.
and your individual spending choices have no bearing on whether the merchants will continue to charge that fee. So you might as well get the benefit of it
EDIT: point was, it's really nothing to do with whether other people are carrying balances or not.