[skohan]

I also wonder if Hyper is really the right tool for the job here. When I was looking into HTTP/S crates, I decided against Hyper because it seemed to require bringing in a runtime for HTTPS, and the async nature of hyper did not seem necessary for my totally synchronous CLI tool. For something like CURL it seems like you would just want the leanest, simplest synchronous HTTP implementation possible

[ordu]

I think that it is the right tool.

1. CURL without https seems insufficient nowadays. 2. CURL could be improved by running multiple downloads at once. I'm not sure that curl command line utility could do it, but certainly libcurl.so has this ability, it allows client code to work with multiple connections. 3. Any application having UI could benefit from async: input/output and main task are async by nature. For example, curl might want to show progress/status to a terminal despite of a stalled connection.

So maybe Hyper is too much of a code for a curl, but it is arguable that it is not.

[skohan]

1. So as to your first point, I totally agree CURL needs to support HTTPS. My point is that Hyper needs a runtime for HTTPS, and it doesn't necessarily make sense for CURL to have a runtime.

2. I'm not sure that CURL should necessarily support multiple concurrent downloads. It could also be argued it's more UNIX-y to make it just do one thing and allow the caller to run multiple CURL processes at the same time

3. You wouldn't necessarily need async to achieve these goals. You could easily have a synchronous http implementation which allows for printing to the console between receiving chunks of data from the network. And if you really didn't want blocking to have a "spinning" activity indicator or something, you could still achieve it with threads.

I think ultimately you'd have to decide based on the relative cost of including an entire runtime vs. just launching a second thread.

[eptcyka]

Hyper only brings in a single-threaded runtime, so it's not much of a runtime at all. That is to say, driving a future returned from Hyper in a blocking fashion and then dropping it is all that's required. Once you drop the client, the runtime will be dropped too. I usually take issue with runtimes due to the added complexity and the lack of clarity about resource usage - I'm mostly worried about superfluous memory usage and some rogue threads going off and doing a quest and a half doing god knows what, hogging or otherwise interfering with my application's threads. I don't think that's possible in this case. Do you know if there are any other concerns that I should be worried about when bringing in something that has a runtime?

[ordu]

> You could easily have a synchronous http implementation which allows for printing to the console between receiving chunks of data from the network. And if you really didn't want blocking to have a "spinning" activity indicator or something, you could still achieve it with threads.

One could do a spinning indicator, but not a label STALLED. To get this one need to restart read(2) every while, and there we come to implementation with complexity on par with async. Things become even more interesting if a program wants to process user input in async. UNIX-way is to send signals, but it is just plain ugly. dd from coreutils allows to use signals to trigger it to print progress, it is very inconvenient way to do it.

> I think ultimately you'd have to decide based on the relative cost of including an entire runtime vs. just launching a second thread.

I'm not so sure. Runtime for user-space context-switching is very small. I did it for educational purposes at some time in the past with C. It is operation like save registers, switch stacks, restore registers and jump to another thread. If you have more than two threads, then you'd need some kind of structure to store all contexts and to decide which one to choose next. Add some I/O code (like epoll) to track state of file descriptors, and you are done. One could do it without async, but it wouldn't become much smaller, because it would be the same logic, just instead of stack switching program would recreate stack frames.

[bagder]

Tadaa: curl already supports --parallel to download many URLs simultaneously...

[mikepurvis]

One better than multiple parallel downloads, aria2 can even be a long-running daemon process that does downloads on demand via a websocket API: https://aria2.github.io/manual/en/html/aria2c.html#rpc-inter...

[tele_ski]

The curl easy api perform call is just a blocking wrapper around the curl multi async interface afaik, so I actually think it makes sense.