Hacker News new | ask | show | jobs
by gsmethells 3277 days ago
Why do I get the feeling this is repeating TCP features at the Message level? There must a protocol that can hide this exactly once need away. TCP doesn't create downloads, generally, that are bad and fail their checksum test, hence packets that make up the file are not duplicated.
2 comments
jkarneges 3277 days ago
Yes there is some duplication of TCP capability here.

The problem with relying on TCP for reliability is that its state is in memory, associated with a particular peer IP address, and acknowledgements passed back to the sender only indicate that the receiver has the data in local memory, not that the data has been processed.

A file download over TCP can fail, for example due to a network problem. Ensuring reliable delivery requires additional measures outside of TCP, such as retrying the download using a new connection.

In practice, this means that TCP is primarily useful for providing flow control and offering a streaming interface (no worry about packet sizes). Less so as a complete solution for transmission reliability.

link
jmaygarden 3277 days ago
How would you use TCP sockets to de-duplicate Kafka streams with a many-to-many communication pattern? Surely there is a valid scalability reason for why AWS IoT only provides "at least once" guarantees in their MQTT broker even when TCP is the underlying transport [1].

[1] http://docs.aws.amazon.com/iot/latest/developerguide/protoco...

link