If everyone is treated poorly by the HOA, you should be able to get them together and dissolve the HOA. Read the documents, there should be a procedure for dissolution.
Depending on the location, the way to dissolve an HOA is nearly impossible. You need to find out what to do with community assets (in a condo situation, this is not possible), and you also usually need somewhere between 75-100% of all members voting 'yes' (not just present voting). If it's 100% of all members, just a single uncooperative board member could prevent dissolution.