Hacker News new | ask | show | jobs
by discreteevent 3265 days ago
In my opinion it's the integration tests that provide 99.99% of the value in 99.99% of the cases. Maybe not full integration tests (which can be expensive to run) but at least subsystem tests. These are the ones that find the real bugs. A very high percentage of unit tests once passing tend to continue to pass and only fail when you intentionally change the implementation. i.e. they impede your progress. Low level unit tests are very good for testing an algorithm or something like a parser, say, but that's not a high percentage of the code on the systems that I have worked on.
4 comments

That matches my intuition. The espousal of unit testing vs other types of tests as "the one true way" actually impeded my adoption of testing in general as I couldn't wrap my head around how to approach it without suffering a huge productivity hit. I thought it was just me "not getting it" but I sense the pendulum is beginning to reverse direction.
I agree, this matches my experience. I find that many bugs can crop up in the interface, which is something that TDD inherently doesn't catch, whereas integration testing will both catch bugs in the interface and the units themselves.
Exactly my experience too. I have been an automation developer in three teams across two games companies - integration tests provide great value if run continuously and daily, they are especially helpful for uncovering regression. Also the parent describing the sad state of affairs in the AAA games industry is fairly accurate and that leaves little hope for TDD/BDD and other similar practices. One of the sources of all of these problems are business model challenges at a company level which then leads to low pay and high stress creating high attrition. Things just aren't planned out the way you may think they would be. The toss away tradition means that unlike companies developing long running services where automated tests written once may be valuable for even a DECADE, that's not true in games so the ROI isn't quite as high, although this may be changing slightly at least when it comes to engine churn and so in does appear to be slowing down.

Aside - having worked on automated testing in the AAA games industry I have found that good reporting helped managers and other devs better see the ROI from the work put in and I have been involved in creating a service for reporting automated test results across dev teams, see my profile if you're interested and give it a try, you can use it for free and I'd love to get your feedback - send me an email.

I agree with that.