Hacker News new | ask | show | jobs
by lukaslalinsky 3480 days ago
> Nothing will happen without you being aware of it

This is exactly the opposite what I understood from the article that Rayon does. For example, it will schedule your code differently depending on the current conditions of the CPU. That kind of non-determinism is hard to deal with if you are doing something more complex than adding numbers. I imagine this is fine for building mathematical libraries, where you care about the result, not about the order of operations. But if you are interacting with external APIs, it's very hard to reason about this kind of parallelism.

4 comments

This makes no sense to me. You are absolutely aware this will happen because you called Rayon's `.par_iter()` method!

The pitfalls of parallel and concurrent computing are well known, and you are correct should not use a data parallelism library for effectful actions whose order matters. But of course the same concerns can arise when using goroutines.

In contrast, Rust actually does guarantee an absence of data races, in rayon or in any other concurrency or parallelism library, whereas Go provides no such guarantee.

I know it's not the same, but in Go you can compile with `-race` and it'll catch race conditions at run-time (useful for testing/debugging).
I think that data parallelism is a very much wrong tool if you are interacting with external APIs – that's in the realm of concurrency and asynchrony. (Check out the work on Futures-rs and Tokio, they are inspiring projects!)

Rayon requires the callbacks it calls to be `Sync`. That means that they are declared to be safe to run across threads. That means that the API it provides explicitly requires the calculations to be parallelizable (and this is compiler checked), and if it doesn't run them in parallel, that can't hurt, right?

Rust can't stop random libraries from providing abstractions like this, but it does allow such libraries to be written to absolutely minimise overhead and maximise performance for when people opt in to using it. If this sort of parallelism isn't appropriate for the task at hand, the language isn't opinionated about what other schemes can work just as well as rayon.
> This is exactly the opposite what I understood from the article that Rayon does

You are aware of it. You put the par_iter() call there. Rust won't magically parallelize regular .iter() loops.

If you don't want to reason about parallelism, don't use Rayon. That's pretty explicit.