[how_gauche]

This was one of my Q4 projects at work last year. We moved CI to 3x hetzner machines, each running four copies of the self-hosted github runner, and drove our build/test times from >20min down to 3-4 min on average. It's ridiculous how big a difference running on a capable bare metal box makes. We run a thousand or so builds daily and pay about 300 euro a month for the setup; our overage fees from github actions were often higher than that. Reliability has been "ok": one of the machines started throwing errors that smell like bad RAM/CPU (bus errors, random reboots, etc), we raised a support ticket, they nuked it and gave us a fresh one.

We provision them with ~200 lines of shell script, which we get away with because they are not running a "prod" workload. Don't forget to run "docker system prune" on a timer! Overall these machines have been mostly unobtrusive and reliable, and the engineers greatly appreciate the order of magnitude reduction in github actions time. I've also noticed that they are writing more automation tooling now since budget anxiety is no longer a factor and the infrastructure is so much faster.