[chimprich]

However, if you have a 1000+ line function that you split into small functions you can pretty easily write a few unit tests per function to see which, if any, of those chunks have problems and then need to be fixed. It's pretty much impossible to write unit tests that can sensibly test a non-trivial 1000+ line function. You might get away with it if it's doing something very straightforward but I wouldn't be very confident in it.

[jblow]

This is fine if you believe that unit testing every 40-line chunk of code is remotely worth the time and effort. I don't think that is true for most applications.

How long does it take you to write and test all those tests? Could you have been doing other things with that time? At 40 lines of functionality, the tests are going to be at least as big as the things you are testing (??), so what kind of a multiplier are you taking just on lines of code written? How much does that cost?

[I run a software company where I pay for the entire burn rate out of my own pocket. So these questions are less academic for me than they are for many people.]

[chimprich]

It's true that there's a non-zero cost for each test, but overall I think tests speed up development rather than slow it down (unless you go crazy with the tests, and providing you're fairly decent at writing tests). I don't believe it's worth testing all paths through the code, but I'd aim for significantly over 50% coverage to have any degree of confidence in the codebase.

I estimate I write about 2:1 unit tests to code in terms of tests to functions but tests should be quite a bit faster to write than the code they're testing. I think I'm at the low end of how much I test my code compared to other engineers, however.

Perhaps it is different in game development. One of the big advantage of writing tests is that you can aggressively refactor with confidence; if you're planning to stop improving your codebase once the game is released maybe this isn't an issue? Plus bugs are perhaps less of an issue if you inconvenience the gamer rather than lose someone cash, and maybe you aren't expecting to hand code over to new developers.

[jblow]

It sounds like you use web languages or other bad programming languages? Aggressive refactoring is not a problem in statically typechecked languages, in fact it is common.

We test the hell out of our stuff, and it works way more reliably than most web sites I have ever seen. But we don't do it with unit tests, because unit tests are not very useful in complex systems, because they do not test anything hard!

[RotsiserMho]

That's quite the blanket statement. Refactoring is just as much of a problem in statically typechecked languages. I write unit tests for a large C++ codebase and find them incredibly valuable when refactoring. Typically each class has a responsibility. The unit tests verify they are carrying out that responsibility correctly. Acceptance tests validate behavior across multiple units (the hard stuff). I fail to see how unit testing in languages like C++ wastes time. We spend far less time finding and fixing bugs than when we didn't have them.

[chimprich]

I've used a lot of languages, both statically and dynamically typed, and found unit testing to be very useful in both.

The idea that unit tests are not very useful in complex systems is very controversial and goes against established best practice in software engineering, the advice of pretty much every authority in the field and empirical studies.

Only testing hard stuff is only half the battle. Unit tests also test basic assumptions and have other benefits like documenting intent. See /Code Complete/ for evidence for the need for multiple angles of testing.

[RotsiserMho]

+1 for documenting intent.