[sirn]

That's strange, in my experience it shouldn't have 6x slowdown. Probably might be due to several factors, but here's your test, running on my system without Docker:

Ryzen 3900X (host machine)

    $ dd if=/dev/urandom bs=4k count=10k | gzip >/dev/null
    10240+0 records in
    10240+0 records out
    41943040 bytes (42 MB, 40 MiB) copied, 1.02284 s, 41.0 MB/s

qemu-aarch64-static

    $ dd if=/dev/urandom bs=4k count=10k | proot -R /tmp/aarch64-alpine -q qemu-aarch64-static sh -c 'gzip >/dev/null'
    10240+0 records in
    10240+0 records out
    41943040 bytes (42 MB, 40 MiB) copied, 3.33964 s, 12.6 MB/s