[SnowflakeOnIce]

About a dozen OSes / configurations were supported, and the entire test suite would take a few days to run on each such configuration.

This was native desktop+server software, not a hosted SaaS thing.

Major releases were put out every few months.

Developers did run tests regularly with every change they would make, but it was infeasible to run all the tests in every configuration for each change. So they would try to choose tests to run that seemed most relevant to the code they had changed.

The entire test suite would run more or less constantly on shared machines, and every few days some new tricky failure would be detected. The tricky failures were almost always the result of some unanticipated interaction of features, frequently on the more obscure configurations (like IBM z/OS).

The problem was not that developers were not testing, but that the tests were infeasible to run every time. So instead, testing became an optimization problem.

[dataflow]

Gotcha, thank you for the explanation! Indeed it sounds like something git bisect is perfect for, when you're in these sorts of situations!