Ask HN: Your experience with a dedicated test team?

[maramono]

Does your team use a dedicated test (non-developer) team? What has your experience been with them? Do they add value? Do you find yourself questioning or being impressed by their skills? Do you use an in-house team or a crowdsourced team (like uTest)?

My own experience has been 50/50.

Some testers are really good and they really like getting deep into the system, learning its intricacies and some are very confortable with code, yet they prefer testing instead of development. These testers show you things of your application that surprise you, and even seem to know the system under test better than you.

But others are really stubborn and seem to want to use the latest fad they learned in the latest testing conference. I discovered that the ones that tend to be bad are the ones that follow three prominent consultants of manual testing: Rex Black, James Bach and Michael Bolton, who really play down all forms of automation, especially BDD.

[JamesBarney]

I think domain knowledge is the most important quality of a good manual tester.

We've had great luck with testers that moved from users to testers. We had a nurse testing as a tester on our clinical app and she was invaluable, we had a geologist testing our seismic applications and she was also invaluable. They also focused on what issues were important in an app, and were great at prioritization.

And we've had cross-domain testers, and they were not as valuable. They seemed to be more nit-picky because they didn't have the domain knowledge to judge what was truly important and what wasn't.

What makes a good automation engineer is pretty similar to what makes a good software engineer.

[greenyoda]

I agree that domain knowledge is essential. Without domain knowledge, testers may spend way too much time testing insignificant scenarios which a user would be very unlikely to perform, while missing obvious problems that every user would run into.

Overall, my experiences with testers has been positive. As a developer under constant time pressure, it's very easy for me to make stupid mistakes that end up causing grief for users. Since unit tests (or any tests written by a developer) can only test scenarios that the developer has actually thought of, it's good that someone else is testing the product from a user's perspective that's not biased by preconceived notions of how the code should behave.

[soulnothing]

I think an auxillary testing group is a good idea. Someone who tests all the components as a cohesive whole. Ensuring it acts as expected in terms of businesss expectations.

My experience though has been counter productive though. I expect a bit of lead time, introducing them to the code base. But I had daily meetings covering the code base, and what it was delivering. Then several weeks of meetings mapping out test cases into a spread sheet, with the various results.

There is a fine balance, but when executed properly they are worth it.

[PaulHoule]

I like having a tester watch my back.

I have been helped by different kinds of testers. I like the guys who keep careful notes, have automation, and can hand the automation code to me so I can reproduce the bug in a split second.

There also was a guy that we hired to do customers who could and would screw anything up and we made him a tester; if he could not screw a system up, nobody could!