> And you are citing this as an example of how to do it right? Why?
Because reference implementations are a bad form of spec. (Note I'm only arguing against normative reference implementations — purely informative ones aren't really a "reference" any more, they're just another implementation, unique in no way except perhaps being the first.)
Why? Because:
- If any disagreements between spec and reference implementation are settled by the reference implementation being right, then there is no motivation to write anything except, "I like unicons". Because, after all, the reference implementation says how it is actually done.
- Similarly, if the reference implementation is treated as being right in case of contradiction, if it has a null-pointer dereference bug leading to a SIGSEGV (per the de-facto standard!), every third-party implementation must SIGSEGV in the same case — because, after all, that's what the spec says you must do!
- Reverse-engineering an implementation is often far harder than understanding a spec, making an independent implementation more difficult to create (if you just want the reference implementation used everywhere, why bother standardizing it?), as one has to get the abstract model out of the reference implementation (which may be implicit and not stated anywhere). This is especially relevant if one wishes to make an implementation using a different model to the reference (e.g., a parallel implementation of something with a sequential reference implementation).
- If everyone uses the reference implementation, the spec (which, in that case, is the reference implementation) is less likely to have so many eye-balls looking over it (looking for inconsistencies, unintended behaviours, ambiguities, and outright bugs) than if it was implemented by multiple, independent people/teams.
Having a spec developed with multiple independent (non-normative) implementations:
- Leads to a clearer spec, having had to be read unambiguously by multiple implementers;
- Leads to greater peer review of implementations, as with a reference implementation it can easily become the only implementation of the spec, thus having to only interoperate with itself, whereas requiring multiple independent implementations requires that they must be able to interoperate; and
- Leads to higher quality implementations, as they are required to match the spec (an external, normative, spec places constraints on the implementations, and therefore leads to better review of the implementations as well as the spec!).
One certainly doesn't want a spec to developed in isolation from any implementation — but one doesn't need a normative reference implementation to garner the advantages of being developed in tandem with one. Multiple independent implementations are needed to ensure multiple people can understand the spec without having to resort to guessing the author's intention (likely differently!).
Ambiguity tends not to be a major issue in a well-written spec, and while certainly a formal spec (such as a reference implementation) does define all cases, it will also define formally all bugs that the author wrote because they are a fallible human. Bugs in specs will occur no matter how formally they are stated and they are a far bigger issue than ambiguity. The important part is to get as much peer review as possible of the spec to maximize the chance the bugs are found — and if you have multiple implementers all reading the spec while implementing it, they are more likely to catch the bugs than someone just taking the reference implementation and using that.
Because reference implementations are a bad form of spec. (Note I'm only arguing against normative reference implementations — purely informative ones aren't really a "reference" any more, they're just another implementation, unique in no way except perhaps being the first.)
Why? Because:
- If any disagreements between spec and reference implementation are settled by the reference implementation being right, then there is no motivation to write anything except, "I like unicons". Because, after all, the reference implementation says how it is actually done.
- Similarly, if the reference implementation is treated as being right in case of contradiction, if it has a null-pointer dereference bug leading to a SIGSEGV (per the de-facto standard!), every third-party implementation must SIGSEGV in the same case — because, after all, that's what the spec says you must do!
- Reverse-engineering an implementation is often far harder than understanding a spec, making an independent implementation more difficult to create (if you just want the reference implementation used everywhere, why bother standardizing it?), as one has to get the abstract model out of the reference implementation (which may be implicit and not stated anywhere). This is especially relevant if one wishes to make an implementation using a different model to the reference (e.g., a parallel implementation of something with a sequential reference implementation).
- If everyone uses the reference implementation, the spec (which, in that case, is the reference implementation) is less likely to have so many eye-balls looking over it (looking for inconsistencies, unintended behaviours, ambiguities, and outright bugs) than if it was implemented by multiple, independent people/teams.
Having a spec developed with multiple independent (non-normative) implementations:
- Leads to a clearer spec, having had to be read unambiguously by multiple implementers;
- Leads to greater peer review of implementations, as with a reference implementation it can easily become the only implementation of the spec, thus having to only interoperate with itself, whereas requiring multiple independent implementations requires that they must be able to interoperate; and
- Leads to higher quality implementations, as they are required to match the spec (an external, normative, spec places constraints on the implementations, and therefore leads to better review of the implementations as well as the spec!).
One certainly doesn't want a spec to developed in isolation from any implementation — but one doesn't need a normative reference implementation to garner the advantages of being developed in tandem with one. Multiple independent implementations are needed to ensure multiple people can understand the spec without having to resort to guessing the author's intention (likely differently!).
Ambiguity tends not to be a major issue in a well-written spec, and while certainly a formal spec (such as a reference implementation) does define all cases, it will also define formally all bugs that the author wrote because they are a fallible human. Bugs in specs will occur no matter how formally they are stated and they are a far bigger issue than ambiguity. The important part is to get as much peer review as possible of the spec to maximize the chance the bugs are found — and if you have multiple implementers all reading the spec while implementing it, they are more likely to catch the bugs than someone just taking the reference implementation and using that.