[electroly]

EC2 instances have dedicated throughput to EBS via Nitro that you lose out on when you run your own EBS equivalent over the regular network. You only get 5Gbps maximum between two EC2 instances in the same AZ that aren't in the same placement group[1], and you're limited by the instance type's general networking throughput. Dedicated throughput to EBS from a typical EC2 instance is multiple times this figure. It's an interesting tradeoff--I assume they must be IOPS-heavy and the throughput is not a concern.

[1] https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/ec2-inst...

[otterley]

That 5Gbps limit is per flow (e.g. TCP connection), not per instance pair. With enough concurrent flows, you can saturate the interface bandwidth between peers, even if it’s 200Gbps or more.

[electroly]

Ah, of course! Thank you for this correction.

[huntaub]

I believe this is also changing with instances that now allow you to adjust the ratio of throughput on the NIC that's dedicated to EBS vs. general network traffic (with the intention, I'm sure, that people would want more EBS throughput than the default).