[Twirrim]

There was a bug in a released kernel I ended up having to hunt down back last summer. It required a very specific set of circumstances to actually trigger the bug, but luckily for us we found some obvious side effects from the bug such that it was possible to get the relevant information via a very simple statically compiled C program (there would be a certain value under /sys).

I used that C program as the initramfs for qemu, with it spinning up an instance based on the test kernel, and could grep for what I wanted in the output to get a relevant exit code.

Combine that git bisect's automatic bisecting process, and I was able to automatically git bisect through a few thousand commits in the kernel to find the cause. It only took it about 50 minutes to find it in the end, which was helped a bunch by ccache (and the sheer build parallelism granted by building on a 52 core system).

That was a fun, somewhat out of the blue, task.

The process I took was based on https://ldpreload.com/blog/git-bisect-run, with various adaptations to suit the needs and the relevant build environment.