But why are card processors annoyed by chargebacks? It's pretty much the reason why these companies exist in the first place, and it should be considered normal part of their life.
Because it's extra work for them. The card company has charged the user, and then sent the money onward to the provider, when the user requests a chargeback, the card company gives the money back to the user, and attempts to take the money back from the provider. This requires human interaction in some cases, and therefore money ... so credit card companies don't want this.