[Jtsummers]

Program logic error:

  // return the maximum value from an array
  return array.min();

Specification error:

  Only users with role X can access this content.
  [where X should've been Y]

In the former case, a test should catch it (because the code isn't doing what we believe, per spec, it should be doing). In the latter case, only validation (confirming with customers) can catch it. Any test that is run against the code will be based on the spec which tells us to do the wrong thing.

Though we usually didn't run into the latter case, ours were mostly embedded systems. So the problem was more like: You implemented this against Industry Standard 1234-A, but we're implementing against Industry Standard 1234-B which says that message Foo word 15 now has a range of 1-20, not 1-19, and 20 means "cease operations", and 18 has been changed from "cease operations" to "halt and catch fire".

So one system was developed per one spec, and another per a slightly different spec. The other common scenario was that the specs weren't created from thin air, but rather based on a preexisting system. And the authors of the spec misinterpreted what the preexisting system did and gave an incorrect specification for a behavior. When testing against the old system (or with the old system as these were mostly communication systems) you'd see a difference in behavior or failure to communicate. But since tests were never truly comprehensive, many of these errors could make it out into the world.

[neatze]

What happens in case where specification is:

[User inputs X1, system displays Y]

On system crash user input was X2.

Is this specification or program logic error ?

[Jtsummers]

I generally consider system stability an assumed part of the specification. Your system should handle most input errors from users more gracefully than a crash. Specifications are never as detailed as the program. So a description of what it should accept implies what it shouldn't. The questions for the programmer when faced with invalid input are:

1. Should it crash? (almost always no)

2. Should it process the garbage input as though it were valid? (pushing the input validation problem further down and potentially causing issues in random locations of the program)

3. Should it reject the input and request a different input? (probably)

Once you get to 3 you've got a number of ways to re-prompt the user or indicate that the input is invalid in a way that won't crash the program. You may need to go back to the customer to figure out their preferred resolution. But crashing is almost certainly not what they want and a sign of a program error (not specification error).

I would classify it as a specification error if they told us X1 would work, and then supplied X2 in a way that was close enough to pass most validation, but not close enough to work correctly.

Like, "The data format is a series of messages. Each message consists of up to 512 16-bit words. Word 0 specifies the length of the message, including itself." and then it turns out that word 0 specifies the length excluding itself causing us to not grab enough data in the first message, and then random amounts of data after that.

[neatze]

If I understand correctly, specification describes "what/when" and program logic describes only "how".