[matthewmacleod]

To be blunt, that means you've not been using it right.

I totally agree that feature or behavioural tests validate that customer requirements are met. But you shouldn't (and probably can't) test more in-dept behaviour in these.

For example, I wrote a shopping basket system for a site last year. There were feature tests in there - "When I click the 'add to basket' button then I should see the item in the basket" sort of thing. Those are great. But I also wrote a whole bunch of unit tests for this - checking that calculations were correctly performed, and the adding and removing items worked correctly, and that tax was applied according to the correct rules, and so on. These tests are super-quick to run and provide a lot of confidence that the API contract is being adhered to. We could have completely switched out the back-end storage for a third-party API or something, and the tests would still be applicable.

There are loads of reasons to test behaviour in layers - I agree that you can easily over-invest in effectively pointless tests, and I've seen that everywhere. But don't discard all unit tests as worthless.

[rpedela]

Based on my very limited understanding of your problem from your brief description, it seems like thorough and properly designed feature tests would have achieved the same thing. In other words, if there is a button in the UI for removing items from the basket, that could have been heavily tested with feature tests instead of unit tests. And those feature tests would serve well if you need to completely rewrite the code at any layer that implements the "remove item from basket" button.

[barries]

There are several issues with relying solely on feature tests.

Feature tests often need to span large parts of an application, so there is a often significant amount of overhead (both code and test-time) in repeating identical-except-for-one-value.

Feature tests often can't test the corner cases of internal code. For example, one hallmark of quality software is that it degrades gracefully in the presence of unexpected inputs. So, while the UI might prevent out-of-range values, programmers often choose to also check value ranges at, for example, the top of a stored procedure. This means that you can't test that code with an app-level feature test because a correct UI won't let you enter values to trigger the stored proc's failure case.

Another big issue is the combinatorial explosion. If you have a processing pipeline, like filters in a sound or image processing app or validation and authorization checks in a line-of-business app, the number of configuration and data values that need to be tested for each stage needs multiply together if you only feature test. Unit testing allows you to make sure each of the stages works "well enough"[1], then you can use far fewer integration and feature tests to make sure that the stages cooperate properly and that system requirements are met (two overlapping, but different, concerns).

[1] However the engineer, team or industry defines "well enough".

[Edit: split up the wall 'o text]

[rpedela]

The internal code argument is weak in my opinion. What does it matter if a stored procedure works 100% if, like you said, that code path will never be executed by the user? There will always be bugs in code so the goal in my view should be to make the application as bug-free as possible from the user's point of view not literally bug-free which is an unattainable goal.

Time-consuming nature of feature tests can be an issue, but often is mitigated by automated testing on commit, merge, etc. But not always of course.

I agree with combinatorial explosion however it can sometimes be mitigated by procedurally generating tests.

[jdlshore]

In a complex app, there are many code paths that are executed by users that are easier to test with unit tests than feature tests. In general, unit tests are easier and faster to write than feature tests if (1) you're experienced writing unit tests (there is a learning curve) and (2) your application design supports good unit tests.

Feature tests are fine for simple or small apps, but I wouldn't rely on them for the bulk of my testing in any significant app.