[molenzwiebel]

For this use-case, you can squeeze out even more performance by using the SHA-1 implementation in Intel ISA-L Crypto [1]. The SHA-1 implementation there allows for multi-buffer hashes, giving you the ability to calculate the hashes for multiple chunks in parallel on a single core. Given that that is basically your usecase, it might be worth considering. I doubt it'll provide much speedup if you're already I/O bound here though.

[1]: https://github.com/intel/isa-l_crypto

[broken_broken_]

Thank you, I will definitely have a look, and update the article if there’s any interesting finding

[secondcoming]

I came across this repo recently and it looks great. It's a pity there doesn't seem to be an official Ubuntu package for it though. There is one for the Intelligent Storage Acceleration Library though.