|
...not only I am being downvoted for asking a simple factual question ("is this how this works, and do I understand the consequences correctly?"), I am also getting obviously incorrect answers. So, let's consider two scenarios: 1) You make a file retrieval request for a 10 GiB file, the front server resends the request to the a single storage server, the single storage server spends 100 ms to locate the file, starts streaming it to the front server, and it takes 10 minutes to transfer it completely; meanwhile, the front server retranslates the data. So you see a 100 ms delay before the file starts streaming, which takes another 10 minutes to complete. 2) You make a file retrieval request for a 10 GiB file, the front server chunks the request and sends them 10 storage servers, each storage server spends 100 ms to locate their chunk of file, then they start streaming it to the front server, and it takes 1 minutes to transfer each chunk completely; meanwhile, the front server waits for all chunks to arrive, which takes 1 minute 100 ms, then sends their concatenation to you, which takes 10 minutes. So you see a 1 minute 100 ms delay before the file starts streaming, which then takes another 10 minutes to complete. Obviously, the latency in the second scenario is worse. Or do I miss some important thing which obvious to everyone else in the sibling comments? |
It doesn’t have to wait for all chunks to arrive, but can start streaming the moment it has the first byte (for some protocols, it may even send you chunks out of order, and start streaming the moment it has _any_ byte)
Also, if throughput from the server to you is higher than that of a single disk, the second case can get the file to you faster. As an extreme example, if that throughput is over ten times that of a single disk, sending the complete file to you can take close to 1 minute.
Having said that, if it has to combine X out of Y > X streams to produce the actual data, getting that will be slower than retrieving a single stream because retrieval times will have some variability.