| I have had great luck using rayon for distributing CPU intensive work across available cores. If you are doing I/O I can see why async is preferable but rayon is an excellent library for parallel work. I have found that async or even in Go with goroutines, a large amount of small threads are not faster even if they should be in theory when it comes to CPU intensive work. I use it to generate a 3D universe (at the atomic level) but store the data as enums. Here is the repo: https://github.com/selfup/oxidizy Big help from rayon: 1) https://github.com/selfup/oxidizy/blob/master/crates/unigen/... 2) https://github.com/selfup/oxidizy/blob/master/crates/unigen/... 3) https://github.com/selfup/oxidizy/blob/master/crates/unigen/... Example perf generating an obscene amount of data on a 32GB machine with 16 threads (3800x): Atoms: 110592000
Baryons: 26099712000
Quarks: 78299136000 real 0m12.779s
user 0m0.000s
sys 0m0.000s |