Did both machines compile to the same target architecture? If you did native compiles, then perhaps LLVM's ARM backend is simply faster than its x86 backend...
I think this is potentially a huge part of it - I'd do the benchmark by doing a native compile and a cross compile of each, and also do the same on a RAM disk instead of SSD (a large compile can end up just being a disk speed test if the CPU is waiting around for the disk to find files).
Maybe that would have been the case if only compilation times were reported to be good. But no, this is across many different kinds of workloads. Even Blender running in Rosetta can beat native x86 processors, which is bonkers.