[sigil]

We opted for an alternate, dynamic approach, which allocates work in real-time using a work queue. We manage all coordination between workers using an nsqd instance... In order to get maximum parallel performance out of our build servers, we run tests in separate processes, allowing each process to make maximum use of the machine's CPU and I/O capability. (We run builds on Amazon's c4.8xlarge instances, which give us 36 cores each.)

This made me long for a unit test framework as simple as:

    $ make -j36 test

Where you've got something like the following:

    $ find tests/

      tests/bin/A
      tests/bin/B
      ...
      tests/input/A
      tests/input/B
      ...
      tests/expected/A
      tests/expected/B
      ...
      tests/output/

    $ cat Makefile

      test : $(shell find tests/bin -type f | sed -e 's@/bin/@/output/@')
      
      tests/output/% : tests/bin/% tests/input/% tests/expected/%
              @ printf "testing [%s] ... " $@
              @ sh -c 'exec $$0 < $$1' $^ > $@
              @ # ...runs tests/bin/% < tests/input/% > tests/output/%
              @ sh -c 'exec cmp -s $$3 $$0' $@ $^ && echo pass || echo fail
              @ # ...runs cmp -s tests/expected/% tests/output/%
     
      clean :
              rm -f tests/output/*

You get test parallelism and efficient use of compute resources "for free" (well, from make -j, because it already has a job queue implementation internally). This setup closely resembles the "rts" unit test approach you'll find in a number of djb-derivative projects.

The defining obstacle for Stripe seems like Ruby interpreter startup time though. I'm not sure how to elegantly handle preforked execution in a Makefile-based approach. Drop me a line if you have ideas or have tackled this in the past, I've got a couple projects stalled out on it.