Hacker News new | ask | show | jobs
by exhilaration 3740 days ago
How does this work? I see the pending POST in the Chrome network tab, which completes when someone downloads it, and then become pending again for the next download. So are you proxying the uploaded data to the downloading client?
1 comments
hakvroot 3740 days ago
That's exactly what's going on. In a bit more detail: when a POST arrives and no receiver is available yet the server lets the sender know that it won't handle any more data (TCP Zero Window). When the receiver connects to the server the server asks the sender to continue writing and the data is relayed (proxied) from sender to receiver. If multiple receivers connect they're put into a queue and the server waits for the sender to POST the data again once the ongoing transfer is finished.

An interesting difference is that in the browser the rePOSTing is done through scripting whereas when using cURL or Wget (or httpie, or ...) the server will issue a 307 Temporary Redirect allowing cURL/Wget to send the file again without manual intervention. Hence the name! (although that doesn't really account for the "Station" part yet)

link
darkf 3739 days ago
Hi! Thanks a lot for sharing details about it, it's an ingenious app. :)

Could you please elaborate a bit on how you work with the TCP window size -- how might I set it to 0 and back? Do you simply use SO_RCVBUF or is there more to it?

If you happen to go through with making more variants of this, it would be amazing if you could make them open source as well (even just the backend). It's a clever idea that solves a very real problem!

link
hakvroot 3737 days ago
Thank you! And I'd be happy to elaborate a bit more, although you already explained most of it yourself ;). Currently the server just stops reading from the socket and the rest is being handled by the networking stack (i.e. buffer fills up, window size is to 0). At this moment the buffer size is also fixed (it sits somewhere between having good throughput and not consuming too much memory for parked connections), but an improvement would be to change it dynamically to improve behavior on both fronts.

For any other versions or source releases I'll post an update!

link