|
|
|
|
|
|
by NovemberWhiskey
817 days ago
|
|
|
This is the thing that I'm finding difficulty with - the formal specification isn't an alternative to an informal specification; it should go along with one. Specification "by example" is, in my experience, almost guaranteed to result in missed cases or unspecified behaviors in uncommon situations. |
|
|
Why are you finding difficulty with that? Not having formal verification is an alternative to having a formal verification.
There ought to be a cost/benefit analysis applied to the tool - i.e. does the cost of writing and maintaining the formal specification pay for itself in terms of bugs caught. Does it have the potential to create new kinds of bugs? (I would argue yes).
The common belief is that it does bring value for certain types of code (rocket ships, maybe pacemakers? etc.), however very few people actually use it because for most applications a certain level of bugginess is perfectly fine. As a result, very few people actually have experience with it.
>Specification "by example" is, in my experience, almost guaranteed to result in missed cases or unspecified behaviors in uncommon situations.
Specification by example does mean missed cases, yes, but the missed cases will get caught by manual testers involved in the process of writing the examples, programmers while implementing the code or fuzz testing.
The dysfunctions I tend to see aren't edge case scenarios being missed altogether, but:
* Not involving programmers / testers who would spot the edge cases in the process of writing the examples. This is typically an organizational dysfunction.
* The programmer decides themselves what the correct behavior should be during edge cases without consulting the stakeholders.
* The PO/PM/Developers make poor decisions about overall architecture or intended edge case behavior. A large part of good system design involves constraining the number of inputs to a system so that the number of potential scenarios doesn't explode unmanageably.
The question I think formal verification has to answer is - does it actually bring any value if you are already doing all of those things or is it more of a performative ritual to ward off the bug spirits?