Hacker News new | ask | show | jobs
by etra0 1470 days ago
Sometime ago I was comparing go, python and Rust to do some GET request asynchronous.

At first, I noticed that the go version was actually faster than the Rust one, and then I saw that in `reqwest`, they recommend you if you're doing multiple GET request, to create a `Client` and then use that to get better performance[1]. After changing my code, the Rust version was effectively a bit faster (not by much, to be honest, which was a bit disappointing considering go's version was way easier to write, and I say this as a generally rust shill).

Hopefully this comment is somewhat helpful :)

[1] https://docs.rs/reqwest/latest/reqwest/#making-a-get-request

4 comments

reqwest::get() is even worse than not having a connection pool. It will also reload the full content of system certificate stores on each invocation - since it creates a new reqwest client. On some hosts that can take 10-100ms alone.

Always create a client explicitly. And also always add a timeout.

The Go http.Get() function uses a shared global client, so making a request doesn't have high initialization costs, and requests can make sure of a shared connection pool.

They recommend you if you're doing multiple GET request, to create a `Client` and then use that to get better performance

Right, then it doesn't have to reopen the connection for each request. That's not an async thing, it's a caching thing.

In heavily IO bound workloads for a compiled language like Rust and Go the bulk of the time will be spent waiting for IO. In that world the optimzations of the compiler for CPU bound operations will fade into the background so it's not suprising that Go is competitive with Rust for that kind of workload. If your workload is this type and Go is equally supported Rust then Go may be a better choice.
Python’s request is exactly the same (the client is called Session). I guess the go client just uses a global connection pool by default?
Yes - `DefaultClient` in `net/http` is what the various package level methods operate on. This is constitutionally bad as global state that dependencies can mutate at will during init (or any other time), hence go-cleanhttp [1].

[1]: https://github.com/hashicorp/go-cleanhttp