| When I say that testing cannot be automated, this is how I work that out in practice: Let's say some developer runs his "unit tests" as part of a build process. Let's say that they are automatically kicked off, as part of the pipeline. This developer may say "I've done testing." I would say, well, we don't know yet. It depends on the answer to these questions: 1. Do you fully understand the unit-level checking code that is being run? Did you write it yourself, for instance, and do you remember what it does and does not do? (IF YES, IT MIGHT BE TESTING, BECAUSE IT'S POSSIBLE THAT YOU COULD MAKE A REASONABLE JUDGMENT ABOUT THE COVERAGE AND VALUE OF THIS PROCESS) 2. Do you fully understand the intent and logic behind the code? In other words, do you understand why the code was written as it was instead of doing something completely different? (IF YES, IT MIGHT BE TESTING, BECAUSE IT'S POSSIBLE THAT YOU COULD MAKE A REASONABLE JUDGMENT ABOUT THE COVERAGE AND VALUE OF THIS PROCESS) 3. Were the automated checks performed correctly? Can you be reasonably sure of this? Did you supervise them in progress or otherwise receive log output that serves as compelling evidence? (IF YES, IT MIGHT BE TESTING, BECAUSE IT'S POSSIBLE THAT YOU COULD MAKE A REASONABLE JUDGMENT ABOUT THE COVERAGE AND VALUE OF THIS PROCESS) 4. Have you reviewed the test results? Did your code collect any metadata or raw data of any kind that may indicate problems beyond those that were the immediate subject of verification? (IF YES, IT MIGHT BE TESTING, BECAUSE IT'S POSSIBLE THAT YOU COULD MAKE A REASONABLE JUDGMENT ABOUT THE COVERAGE AND VALUE OF THIS PROCESS) So, a checking process becomes testing when a tester applies his judgement to the situation. Otherwise, it's just checking in a vacuum. Oh, and I'm only talking about the situation where everything is "green." If it's not green, then for testing to be happening, someone must investigate. An example of checking that is not testing is when my car's "check engine" light comes on. I am not qualified to interpret this event, other than to take me care to a repair shop. When the repair guy looks at it, that check becomes a test, and then he acts on the result of that test. |
What is "testing?" Well James (or Cem or . . .) will proclaim the definition in an enthusiastic and self-assured way and then tell you how you're not doing their just-made-up definition. It feels a bit like a self-help seminar. And I'm not just saying that because you also have books available in the lobby.
And the specific claims are over the top, too. "If you didn't watch your 'test' run, it may not have been 'testing'" isn't very helpful. Especially when you're talking about dev unit tests.