Hacker News new | ask | show | jobs
by sophe 3095 days ago
The class of tool you are looking for is a Test Case Management tool (TCM). The basic model of a TCM is a db for holding atomic test cases. Most TCMs let you collect test cases into "test runs", which is a set of test cases to be executed in a context. Test runs can typically be organized into some kind of project bucket (e.g., "drop 11" or "release foo").

TCMs usually support tagging of test cases, so you could for example tag some test cases as "smoke test" and collect them into a smoke test run.

TCMs usually support the manual execution of test cases: the tester works from a "test run" and sets status of that test cases instance to pass/fail/etc. Depending on the state of your test automation, you can also do the same status setting for an automated "test run". Some TCMs support the automatic updating of a test case instance via APIs triggered by integration with test frameworks.

Some TCMs support integrations with bug trackers, for example with Jira. This means you could create a virtuous circle of automated test execution, test run status updating, and bug creation for failed tests, depending on the maturity of your test automation infrastructure and your TCM and bug tracker tool choices.

My current preference is for Testrail (http://www.gurock.com/testrail/) over TestLink.

But here's the deal: a TCM is a far better tool than Word or Excel in tracking test cases or test execution, but TCMs come with gotchas. Any QA or test effort is a dynamic process that should evolve over time. It's easy to put way too much emphasis on the TCM as the goal of your test work, but it's just a tool that supports a process. Don't use a TCM as your strategic test planning tool, because it's a software product written by external teams according to their own crazy software development realities and definitions of process and quality. Figure out what you need to focus on, what your workflow should be, and use a TCM as a tool that helps you right now, and drop it if you find yourself changing your work patterns to suit the tool.