[tomkwong]

>You've just introduced an exponential explosion into your testing burden. The pay-off: additional functionality that might be useful later. > edit On reflection I suppose it's not a 'exponential explosion', it's merely a 'multiplicative explosion'.

This statement seems rather inaccurate and misleading. What you thought about 'multiplicative explosion' is the number of integration tests required. Generally, software testing focus more on unit tests, and so in this case, the number of tests is still linear.

> The article doesn't go into detail on what it means here, but the more serious the software work, the less the artistic mindset applies. [1]

This claim seems to be too opinionated. I doubt seriousness makes a difference. I have seen artistic/creative design in some very serious projects before.

[MaxBarraclough]

> What you thought about 'multiplicative explosion' is the number of integration tests required. Generally, software testing focus more on unit tests, and so in this case, the number of tests is still linear.

I don't follow. The new function has 9 different modes of operation to be tested. I don't see the relevance of the distinction between unit tests and integration tests.

The alternative is to decline to test them all, of course, risking a trade-off between bugs and baggage.

Even in this trivial example, there's a bug resulting from inadequate testing. [0] Perhaps in this example we might be confident enough in the independence of the two parameters that we wouldn't bother testing all 9 combinations, but my general point stands.

> I have seen artistic/creative design in some very serious projects before.

What kinds of projects? Avionics? Formally verified systems?

Formal development solutions like SPARK, and informal frameworks like MISRA C and other restrictive coding standards like [1], greatly restrict the programmer's freedom.

[0] https://news.ycombinator.com/item?id=23365655

[1] https://news.ycombinator.com/item?id=22192656

[xg15]

> I don't follow. The new function has 9 different modes of operation to be tested. I don't see the relevance of the distinction between unit tests and integration tests.

The alternative is to decline to test them all, of course, risking a trade-off between bugs and baggage.

Even in this trivial example, there's a bug resulting from inadequate testing. [0] Perhaps in this example we might be confident enough in the independence of the two parameters that we wouldn't bother testing all 9 combinations, but my general point stands.

I think what he means is that, in unit testing, you only test code units in isolation to each other, with the tests assuming the other parts that your unit-under-test interacts with behave exactly as the unit-under-test expect: If you had n producers that could plug into one of m consumers, you'd test each producer in isolation against a mocked consumer and each consumer in isolation against a mocked producer. That will give you n + m unit tests, which is indeed linear.

Of course this hides the fact that in practice, you'll have n * m different modes of operation, so you'd need n * m different integration tests if you actually wanted to test each possible interaction.

You can get away with only testing units in isolation if the interface/API/protocol between producers and consumers (in this example) is well-defined and rigid enough that implementation details in one part really have no effects on the other part. I think there is lively debate how well this is possible in practice - in effect, this is the whole point of standardized interfaces. However, even attempting this is often a lot harder than it appears and therefore shouldn't be done if not necessary.

That being said, I think it's important to keep in mind that you can almost never write tests for all possible configurations that a program could run in, if your program is non-trivial. You get combinatorial explosion pretty fast, even if you don't use any abstraction at all. And after all, the whole point of a program is that you can feed it with novel inputs without knowing what the exact output will be - becasue finding out that output is what you wrote the program for. So you'll always have to think about classes of inputs and restrict your tests to (hopefully representative) examples of each class.