[slt2021]

my take is that within-datacenter traffic is best served by Ethernet.

Anything on top of Ethernet, and we no longer know where this host is located (because of software defined networking). Could be next rack server, or could be something in the cloud, could be third party service.

And that's a feature, not a bug: because everything speaks TCP: we can arbitrarily cut and slice network just by changing packet forwarding rules. We can partition network however we want.

We could have a single global IP space shared by cloud, dc, campus networks, or could have Christmas Tree of NATs.

as soon as you introduce something other than TCP to the mix, now you will have gateways: chokepoints where traffic will have to be translated TCP<->Homa and I don't want to be a person troubleshooting a problem at the intersection of TCP and Homa.

in my opinion, the lowest level Ethernet should try its best to mirror the actual physical signal flow. Anything on top becomes software-network network

[mafuy]

In data centers/HPC, you need to know which data is flowing where and then you design the hardware around that. Not the other way around. What you describe is a lower requirement level that is much easier to handle.

[marcosdumay]

That may be true for HPC, but "datacenter" is a generic name that applies to all kinds of structures.

[gsich]

>Anything on top of Ethernet, and we no longer know where this host is located (because of software defined networking). Could be next rack server, or could be something in the cloud, could be third party service.

Ping it and you can at least deduce where it's not.