| I apply the following reasoning model to testing: - Integration tests are expensive to run and take time to write; therefore it is important to maximize their value. The ultimate integration test is an end to end test because it maximizes the scope what is under test and the potential for weird feature interactions to trigger exactly those kind of failures you want to find with such a test. - Unit tests are orders of magnitude cheaper to run; so have lots of them but make sure they are easy to maintain and simple so they minimize time spent on them. - Anything in between is a compromise between shooting for realism vs. execution speed. Still expensive to run and maintain but it just does not deliver a lot of value. - Test coverage becomes exponentially harder with the size of the unit you are testing. Test coverage for integration tests is a meaningless notion. With end to end integration tests you shoot for realism, not coverage. They should cover things that users of your system would use in ways that they would use them. - Mocking and faking is needed to unit test code that violates the SOLID principles and is otherwise hard to test. So they have the development overhead of an integration test but they deliver the value of a unit test. This is not ideal. It's better to unit test code that is very testable and cover the rest with integration tests that deliver more valuable insight. Lots of very complex unit tests are hard to develop and limited in value. I just removed the one remaining test that used mockk in my Kotlin code base. I have hundreds of API integration tests. And lots of simple unit tests. I focus my unit tests on algorithms, parameters, and those sort of things. My integration tests ensure the system does what it needs to. I run integration tests concurrently so they complete quickly. This increases their value because it proves the code still works if there's more than one user in the system. |