[datr]

That's true but it would seem like there would be other disadvantages to.

For example, when do you do testing? If we test as soon as the pull request is opened, we know that master is going to change a lot between now and when we finally deploy this code so the tests might not be valid.

If, on the other hand, we wait to test just before we deploy to live we risk locking up the queue for too long. This might not be an issue if you're test only take a couple of minutes to run but if you have lots of integration tests (like facebook for example [1]) then it could become a big issue.

Is the solution to this, you just accept that the codebase you're testing won't exactly be the same as what's deployed to production and the risk that comes with that?

[1] https://developers.facebooklive.com/videos/561/big-code-deve...