[pm215]

This is the same trick used by the old 'scratchbox' development environment that Nokia used for the maemo SDK. If the stuff you compile in it includes classic "configure and make" C/C++ projects then you'll probably also find that replacing the /bin/sh binary with an x86 one improves speeds -- configure and make spend quite a lot of time spawning shells to run bits of shell script.

[valkmit]

Hey, thanks for the insight - I didn’t realize this was a thing for some Nokia envs already - pretty cool!

re: x86 make and shell, it probably won’t work well here because once a process tree goes to x86 land, qemu cannot see it and it can’t “natively” run an arm binary.

Essentially we can only hook execve when emulating - when we run a native binary we no longer have any insight into any execve call it may make. (Though with some additional work I suppose one could also use LD_PRELOAD to hook execve)

This is why binfmt_misc and registering the interpreter with the kernel would be superior, if not for kernel level access requirement

[pm215]

QEMU doesn't hook execve anyway, at least not upstream -- when a QEMU emulated process invokes another we rely on the binfmt-misc magic to cause the kernel to run QEMU on the child. We don't special case "the binary being exec'd seems to also be for the guest".

It looks like maybe you've made some local patches to QEMU that relate to execve?

[bsdimp]

I know the qemu-bsd-user fork has some hacks for the bsd-user stuff that looks at the binary to be exec'd and does different things based on the target since there's some optimizations you can get out of this if you don't rely on the kernel 100%... Bits of them may even be in my latest patch-set :)

[valkmit]

Cool stuff! Do you have a public branch you're working off of? I'd love to follow the progress as I wasn't aware that qemu user emulation for BSD was even a thing.

I didn't get a chance to peruse beyond briefly skimming the source on GH, but I couldn't find any special handling around execve - could you point me in the right direction?

I was looking at bsd-user/syscall.c, whose equivalent in linux-user is where my repo's execve hook lives, if that makes any difference.

[valkmit]

Yep! I have a patch to qemu based on one that never made it to master upstream. It’s a sub module in the repo