[mhandley]

Using user terminals was as much of a thought experiment as anything. The graphs in the video show you can do pretty well just using a few well placed groundstations - you can shave another millisecond or three off with higher densities, but it's not a lot. The big advantage of higher densities is really that you can get better spatial reuse - with more possible choices you can use RF uplinks in places where the spectrum would otherwise go unused, whereas you've less freedom to choose if you've fewer groundstations.

Having said that, this kind of wide-area low-latency bounced routing is never going to be used for you or me to watch Netflix. It will be reserved for high paying customers who really really care about latency. For you and me, we'll be dumped into the terestrial network at the nearest possible location that isn't already saturated.

The second generation of satellites should have optical inter-satellite links, and then you would only use ground relays rarely.

[portillo]

Hi Mark,

Thanks for your reply. I agree with you, using user-terminals is extremely challenging, especially from a link-budget perspective. You cannot pump enough data to make it worth it. For the gateways, my main concern is that the Ka-band spectrum would have to be shared between user-data and "inter-satellite" data.

Finally, I think that the use-case your described for those latency-sensitive customers is going to be hard to pull off, mainly because of link-availability concerns. There are too many "passes" through the atmosphere to guarantee the availability numbers that a user of such a service requires (99.5%?). Rain in any of these links might cause an outage or a re-route (causing too high jitter). Plus, it would be extremely difficult to have signals traveling from one continent to another.

[mhandley]

On routing: the best path in this sort of network changes something like every ten seconds or so. To make it work, you need to consider all the links that can delivery good enough SNR, so you're already factoring in rain for example. You've got many many possible paths, and you're considering the best route constantly, but you're only factoring links with acceptable quality into the computation. If you do this (and can solve the queue avoidance issues too), turns out jitter isn't such a big problem. As one path gets longer, it eventually gets longer than the next path, and you switch to the new path as close to that moment as you can. If you do this, you minimize step changes in latency. It only works if you've got enough satellites, but it looks like SpaceX will have.

If you can't eliminate all queuing delays, you can't factor all delays into your decision of when to change routes, and so you will get some jitter and hence reordering. If so, you need a reorder queue in the final receiving groundstation, so as to avoid confusing TCP. This removes jitter at the expense of adding some latency. How much latency depends on the queuing delays you're trying to smooth out, so it all really comes down to avoiding queues in the satellites.

[shaklee3]

Completely agree. While the video is awesome (nice job!), the logistics of pulling something like this off is extremely difficult. It's different from the "you could have said that about landing a rocket", in that there are thousands of different routes taken for different customers per second, and each route has its own issues like portillo said. The minute one stops working, debugging that will be a huge challenge.

The user terminal as a gateway idea is also not practical due to link budget (EIRP and G/T), but also the much lower availability they'll be dealing with.

[mhandley]

Computationally, it's difficult, but I don't think it's intractable. Bear in mind this this is not an animation, but a real-time simulation:

https://youtu.be/m05abdGSOxY?t=428

I'm running Dijkstra across this mesh at 30fps in real-time on my laptop while also doing the 3d animation. My laptop fan does spin a bit, but it's not crazily optimized code, and for the video I was also recording to H.264 simultaneously. Doing routing for all customers simultaneously is certainly feasible if their groundstations do the computation, based on routing state supplied in real-time by the constellation. Other solutions are probably possible too, but this seems simplest to me, and scales linearly with customers.

[shaklee3]

While that's true for a first approximation, weather, gateway outages (very common as you increase the number of them), total satellite capacity (making sure you aren't overloading a single link), and just non-working paths are commonplace. Just making sure that you aren't overloading a particular satellite is an extremely difficult problem. That would be really neat if you could work that into your simulation, where it would bypass certain satellites if there's just no more bandwidth available.