[veyron]

"Suffice to say, there were/are pros and cons to having a specialist."

Let's not go there. Specialists were accused of favoritism: filling orders that certain traders submitted and ignoring requests from other traders.

More generally, the advancements, both in technology and in pricing, actually helped everyone. SOES (small order entry system) helped smaller traders to quickly trade. Decimalization really helped reduce transactions costs (spread between bid and offer is a sort of transaction cost, when we were dealing with 1/16th quoting, the spread cost was over 6 cents)

On another front, brokers used to screw people over by telling people they bought at one price while actually buying at a lower price. This is why Regulation NMS came about. And HFTs nowadays help keep all of the exchange prices in line, minimizing the potential damage of shady broker behavior.

[guimarin]

yes and sophisticated HFT systems screw large funds from moving into and out of positions with relative ease/fairness. This cuts both ways, thus pros and cons. At least with a specialist, everyone was relatively even in that regard. Now, measuring the distance to the server in micro-seconds of fiberoptic wires, I can literally create an unfair system by owning the closest supercomputer. I disagree with your point and think that my 'compromise' tax is pretty damn fair considering. Also, I'm not arguing about the merits of digitizing the exchanges, I'm arguing that there was a cost to removing the specialists, it's obvious now, and we should fix it. The specialists were bad in some cases, enough to get people to change, but the system is just as corrupt now as it was then, just differently.

[veyron]

"yes and sophisticated HFT systems screw large funds from moving into and out of positions with relative ease/fairness." <-- that's completely false. It is precisely because of HFT systems that the capital markets were reasonably resilient, and it is because of the lack of HFT in the current environment that prices swing on the order of 5% during the day

"At least with a specialist, everyone was relatively even in that regard." <-- again, specialists were accused of playing favorites. That's manifestly unfair.

"I can literally create an unfair system by owning the closest supercomputer." <-- That's also not true. It's not good enough to have the lowest latency connection. I can have a slightly slower connection but still have an edge. To put numbers to this, lets say that I was 100 microseconds slower than the fastest person, but I could predict the next price movement 150 microseconds faster (this is due to many factors, including the particular model I use and other implementation details). Then, I would still have a 50 microsecond edge.

"my 'compromise' tax is pretty damn fair considering" <-- this would absolutely screw large funds. As was mentioned by many others, if you tax transactions, market makers will quote wider spreads (and yes, spreads are transactions costs to the large funds moving in and out of positions)

"I'm arguing that there was a cost to removing the specialists, it's obvious now, and we should fix it." <-- NYSE parity (which is a vestige of the specialists) is broken and unfair. Why should someone who sent an order after me get filled before me? Most exchanges have a price-time priority concept, so that if we are sending at the same price but I sent first, then I should get my fill first. The specialists wrecked the idea back then and the current implementation is still unfair.

"the system is just as corrupt now as it was then, just differently." <-- it's actually much less corrupt. I recommend you go and try to build your own HFT, and then you will see that the system now is much more fair than you think. (shameless plug: I started blogging about my experience: http://news.ycombinator.com/item?id=2835656 )

[yummyfajitas]

yes and sophisticated HFT systems screw large funds from moving into and out of positions with relative ease/fairness.

Fact: when you make a large trade, there is a price impact. I.e., if you sell lots of GOOG, the price goes down.

Once upon a time, large funds could use their own sophisticated algorithms to make sure their unsophisticated counterparties feel the price impact. I.e., the big fund disguises their intent, Joe IRA buys from the large fund, and then the price goes down after Joe already owns the stock.

Nowadays, HFT makes the price impact happen immediately, typically splitting the difference with Joe [1].

Why is this worse?

[1] The HFT's try to outbid the large fund in order get ahead of it, resulting in Joe getting a better price.