[an-unknown]

It seems like this tool does not create a fully deterministic nor reproducible environment. Hermit seems to only intercept and modify syscalls, but this is not the only source of non-determinism and randomness. For example, the layout of environment variables in memory also causes non-determinism, caused by the content of the environment variables as well as their order in memory. CPU instructions like RDTSC, RDRAND, RDSEED and similar also introduce randomness. It seems like Hermit ignores some these sources of randomness, but I can't test it, because it doesn't build on a current Arch system with the Rust toolchain from the repo.

At least it seems Hermit masks RDRAND and RDSEED via CPUID, but not every program is written to support ancient architectures which didn't support these instructions and therefore not every program tests availability via CPUID.

In addition, even if all of this was deterministic, CPU flags set by various instructions with "undefined" flags according to the CPU manual can slightly differ between different microarchitectures. A "normal" program should not be influenced by this, but it is still a source of non-reproducibility. This might be relevant for certain rare compiler bugs.

[roca]

In practice, those "undefined flags" are set consistently. In Pernosco we move rr recordings across architectures quite often and those flags have never been a problem.