[polskibus]

Google should investigate (or perhaps just buy outright) a low level communications technology stack from one of the HFT firms - they've already mastered low-latency networking, they just have no incentive to share this knowledge with the outside world.

[chollida1]

I think you'd be a bit disappointed with what HFT firms do.

They are limited in what they can do because they have to talk to the exchange, so its still tcp/ip for order sending, with either FIX or a binary protocol like ITCH/OUCH on top.

as far as their networking stack, if they are ultra low HFT then they'll use FPGA's and Arista brand switches or Infinband hardware.

The only big customization that most HFT firms do is move the networking stack into user land, but that's a well known area. I"m not aware of any HFT firms that write their own networking stack from the ground up though I'm sure there are a few:)

Not much that they do is transferable to everyday computing because most, I'd say 90%, of the performance comes from custom hardware and not the software.

Or put another way, Google already has more than enough talent to optimize their QUIC protocol, buying a HFT firm wouldn't do much for them as the HFT speed comes from area's that most people setting up servers won't want touch.

[_delirium]

I think Google's solving a pretty different problem, low-latency communications over a quite heterogeneous network, where they don't control the lower-level infrastructure. HFT firms typically have a narrow range of configurations they have to work over and control the setup of their pipe; they aren't trying to ship technology that will work over every random person's DSL line and funky NAT setup.

[JoshTriplett]

Those communication stacks are not suitable for general-purpose use; they sacrifice everything, including usability, robustness, portability, and a hundred other factors in favor of latency.

For example, such stacks often put the entire communication stack in userspace, with hardcoded knowledge of how to talk to a specific hardware networking stack, and no ability to cooperate with other clients on the same hardware.

[polskibus]

There are plenty of vendors that provide good UDP-based solutions, for example TIBCO. In my opinion multicast is not used widely enough, partially because everybody thinks that tcpip pub sub is good enough.

Financial incentives made HFT's and alike go farther than the average software companies - just look at the microwave networks.

[tptacek]

Multicast isn't used widely for a lot of reasons, but the most important of them is that it simply doesn't scale across routing domains.

[polskibus]

first 3 upvotes, then lots of downvotes - what's wrong with my comment? Why is it so bad to advise buying companies that might have the edge over google? I actually value the comments because they pointed out most HFTs probably do not have the it.

[brighteyes]

There is a higher than usual amount of downvoting in this discussion, with several decent comments getting buried. I'm not sure what's going on.