Hacker News new | ask | show | jobs
by IgorPartola 4197 days ago
I generally don't write unit tests, and very rarely write system tests. Partly this is because of the type of stuff I deal with (Django applications), partly it's because I generally have access to good QA people, but mostly it's because the kinds of errors I see would not be caught by the kind of testing I can write in any reasonable amount of time. Here are some real world recent errors I've had to deal with, where testing wouldn't have saved me:

- Poorly documented external API returns a redirect instead of a 404 in some cases.

- Missing <title> tag in a random HTML template.

- Poorly assembled SSL certificate chain for an HTTPS service fails for some browsers, but not all.

- Celery tasks taking longer to process because there are now more things to do as more users sign up.

- Starting a DB transaction in the wrong place causes more failures than necessary when only some of the modified rows cause constraint violations.

- External API returns 0 sized file with no errors instead of the correct response, depending on time of day and phase of the moon.

- CSS issue rooted in poorly set up position and z-index properties causes elements to be mis-aligned.

- Missing clear: both;

- Database server's disk filled up with log files from a rogue service.

- Hosting provider gets DDoS'ed and their mitigation software starts returning random site redirects.

- Namecheap's DNS gets DDoS'ed and the site is inaccessible.

These are not the types of things that are easy to test, but very easy to verify by hand or catch at runtime. Personally, I am much more in favor of design-by-contract and logging and alerting of bad conditions at runtime. That's not to say I wouldn't test a financial trading algorithm, or a binary search library function. One of my projects (LogHog, a much easier to use syslog-type-thing for Python), has lots of tests because it's got a very simple and well-defined interface. It's almost library code and verifying its correctness is both easy and useful at the same time. But not all projects are like that, and sometimes easy testing is not actually useful and useful testing is cost-prohibitive.

tl;dr: I can test the obvious stuff, but I can also just spend more time writing it more carefully. It's the non-obvious stuff that breaks: disks fill up, external API's misbehave, bad CSS, etc. You will never have "100% code coverage" because you are not testing all the stuff that actually affects your system, and your users don't care why the application is not working, they just care that it's not.

2 comments

I generally agree with you.

"sometimes easy testing is not actually useful and useful testing is cost-prohibitive."

While I agree here, too, I think that the situation where testing is not useful happens very rarely in practice. Everything can break, and when you think it cannot be possibly be wrong now, a future regression might occur.

I think badly written tests are a different problem. I.e. tests that for example assert on the serialized string and hence take a lot of maintainability effort to adapt to changing production code. But then there is nothing wrong with the test per se, its just badly implemented.

I can think of lots of cases where testing is not useful in practice. Simulating random filesystem corruption when writing a Node.js app is pretty much useless: you don't know which files will be corrupted, and testing would take a huge (billions of lifetimes) amount of time to simulate all possible combinations even for a small program. Yet, this can (and in my experience did) take down a production system.

My other examples I already listed, I think, are examples of this too: you can test all you want against a spec provided by your external API maintainer, but if they don't follow the spec, you have a problem. We don't have a good framework (and I don't think we can create one), for testing layout problems in HTML that are caused by bad JS or CSS.

Basically, you do hit diminishing returns quickly, and you do get a false sense of security by having lots of small unit tests that don't actually prevent realistic failures.

As it's an Eclipse plugin I would guess it's aimed at JEE / enterprisey developers.

I code similar apps to that which you describe and tend to agree that up-front testing is hard to justify a lot of the time. However when some obscure but does arise I do try to write system tests (selenium, webrunnners etc) or beefing up the operational monitoring so I can avoid them recurring.