[RossBencina]

The Jane Street podcast was a very approachable explanation, and was interesting to me for a few reasons: (1) that EU regulation requires their timestamps to be within 100 microseconds of UTC, and (2) they claim to achieve 20 microsecond accuracy using hardware-timestamped NTP synched to local NTP time servers, the NTP servers then being synced to off-the-shelf GPS masters using PTP, (3) they really didn't like the idea of running PTP on all of their switching infrastructure (even though running PTP boundary clocks on all of your switches is the obvious way to go), (4) they weren't doing anything fancy or hardcore. Very pragmatic. I was expecting lasers, custom FPGA systems, and a dev team dedicated to time synchronisation.

They did mention that to go below 20 microseconds they'd need a different approach. And mentioned white rabbit https://ohwr.org/projects/white-rabbit/ From NTP to white-rabbit sounds like a big jump to me.

In the context of digital audio, 20 microseconds is an entire sample period at 48kHz. AVB using gPTP is capable of locking up all devices on the network to some small fraction of a sample period. That requires all network switches to propagate time information. Start here: https://en.wikipedia.org/wiki/Time-Sensitive_Networking

[pclmulqdq]

I was personally pretty surprised at the idea that Jane Street found PTP to be too difficult to administer or run. Hardware support for PTP is nearly ubiquitous in NICs and switches (unless you build this gear for yourself, I guess), so it is not that hard to administer. PTP done poorly gets you ~100 ns time sync across your cluster, and if you do everything correctly you can get time within about 10 ns given how small a trading network is.

[AdamN]

This time accuracy would need to propagate to all their hosts, not just the ones in a single DC. I presume they have hosts in the EU, London, NYC/NJ, Tokyo, Chicago, etc... I imagine 100ns accuracy with that kind of global installation diversity isn't straightforward.

[pclmulqdq]

The gold standard for PTP is to use separate GPS-disciplined atomic clocks in your separate points of presence, and PTP within each one. These are about $10k each, so it is not that expensive even if you have 100 datacenters.

[namibj]

Yes it is, GPS is 1foot ~ 1ns; thus a few ns are trivial with a vaguely decent receiver.

[hermitdev]

> that EU regulation requires their timestamps to be within 100 microseconds of UTC,

It's the same in the US. It's covered under CAT NMS (Consolidated Audit Trail, National Market System). Probably too much information at: https://www.catnmsplan.com/

[jeffbee]

Yes, and the fact that this has been a global regulation for years suggests that it isn't all that special. Outfits like Goldman Sachs meet this regulation with fewer blogs.