Not anybody in particular but it does hurt society as a whole. It's one of those things that make the rich richer for no reason other than them already being rich.
This is fair to say, but HFT is a very small industry. The number of people getting "rich" off of HFT is less than 100. The rest of us are just comfortably compensated tech workers, same as a Google or FB engineer