| > End to end tests seem to be a crutch. Generally I agree. If you have good unit tests that fail when behaviour changes, your end to end tests do not need to be run all the time (or, often, ever). The program as a whole operates in certain ways. Ideally, its operation is governed in a deterministic way by the behaviour of its parts. End to end tests usually can't give you full test coverage because the number of permutations of behaviour is too high to specify. However, the behaviour of single units can be exercised fully in most cases -- especially if you write the code to be easy to test (factor out branch points and loops, expose state so that you can test for changes in state, etc). If the behaviour of the units don't change, the behaviour of the final program also won't change. This means that you only have to test the behaviour of the full program in relation to the change of behaviour of the units. Often this can be done with manual testing while the units are being modified. In other words, individual stories need to have acceptance tests (which can be manual or automated), but they usually only have to run when merging into the trunk branch. You don't usually have to run them again on deployment (or on any subsequent branch merge -- as that branch will have its own acceptance tests). Some end to end tests might be a good idea depending on the architecture of your system, or the cost of failure. For example, if you have several disparate systems that are coordinating (e.g. lots of micro services), then it's pretty hard to do acceptance testing when you are doing development. It's one of the massive downsides to doing development that way. Additionally, it's usually a good idea to have end to end tests if a user's safety could be compromised. Or possibly making sure your "Buy Now" button works in various different scenarios. Relying on end to end tests for your entire application is usually a losing proposition because of the burden to get acceptable test coverage and fast development turnaround. |