[msluyter]

Here I'd distinguish between system/integration and unit tests. Unit tests as a whole tend to amount to a mirror of the code base. If a given function f returns '17' and a test validates that fact, all we've done is double check our work -- which has some value, but doesn't protect against the case in which f is _supposed_ to return 18 and both the code and the test are wrong.

OTOH, system tests provide a realm where external implicit/explicit requirements may actually be validated. Perhaps.

[fao_]

IMO tests should never do this unless the function's role is obvious. This prevents internal restructuring (Which is bad!). What you should instead prefer is end-to-end testing.

So for a lexer, you'd create a dummy program with the output of each token on a newline, and then test that the tokenizer's output matches what you expect from the input. But you shouldn't test whether the functions themselves are correct. The tests cases should be designed to throw up any bugs in any of the internal functions, and the system should catch it as defined.

Otherwise you end up documenting what the system currently is, rather than that the goal of the system is met.

[alien_at_work]

Completely disagree here. If you test this way you'll be sure to have a correct implementation for this dummy program. Further, if the test fails you now have to try and figure out what component caused the failure.

A unit test should test a specific unit; commonly a class. You should have unit tests for the interface of that unit (never private or even protected methods). This absolutely does not prevent internal restructuring but it drive you toward seeing your classes as individual service providers with an interface.

[fao_]

> If you test this way you'll be sure to have a correct implementation for this dummy program.

That's the point. If the dummy program fails then your implementation is bad.

> Further, if the test fails you now have to try and figure out what component caused the failure.

Have you heard of logging?

> A unit test should test a specific unit; commonly a class. You should have unit tests for the interface of that unit (never private or even protected methods).

This is almost exactly what I suggested. I stated 'dummy programs' for individual tests because it's more modular and closer to actual usage than mocking. If your language has an equivalent, use that.

[alien_at_work]

>Have you heard of logging?

So your solution is to dig through log files to find the problem instead of read the unit test name(s) that demonstrate the failure?

>This is almost exactly what I suggested.

Well, it wasn't clear from your wording. It sounded like you were advocating integration testing instead of unit testing (which is a thing that people commonly do, so that's why I reacted to it).