That would seem like a logical solution. So wouldn't it be convenient for the expensive payment methods if legalities prevented merchants from charging higher fees to customers using them?
Indeed. It's a triumph of consumer protection laws failing to protect consumers. Merchants here have to set their prices a bit higher to compensate for the fees and you still have to pay those higher prices as a customer even if you're using a more efficient payment method. I will never understand why the law wasn't set the other way - requiring explicit disclosure of payment fees to end customers and prohibiting payment services from incorporating these kinds of anticompetitive terms in merchant agreements - so that everyone could make an informed choice and market pressures would push the transaction overheads down.
It might have been regulatory capture - though I have seen no specific evidence of that myself. It might simply have been the old story about a road and good intentions. At this point it doesn't really matter how it happened - it would be better if the situation were fixed in any case.
In the UK it’s the system the law imposes on everyone.