[rzzzt]

The DoD's 2167 standard from the late '80s mentions the following documentation that should be produced as part of the development process (section 6.2 and Appendix D):

- System/Segment Specification

- Software Development Plan

- Software Configuration Management Plan

- Software Quality Evaluation Plan

- Software Requirements Specification

- Interface Requirements Specification

- Software Standards and Procedures Manual

- Software Top Level Design Document

- Software Detailed Design Document

- Interface Design Document

- Data Base Design Document

- Software Product Specification

- Version Description Document

- Software Test Plan

- Software Test Description

- Software Test Procedure

- Software Test Report

- Computer Sytem Operator's Manual

- Software User's Manual

- Computer System Diagnostic Manual

- Software Programmer's Manual

- Firmware Support Manual

- Operational Concept Document

- Computer Resources Integrated Support Document

- Configuration Management Plan

- Engineering Change Proposal

- Specification Change Notice

[ebiester]

This is a particular artifact of the government system process. These are contracted pieces of work that Company A would deliver, Company B would administer, and Company C would be contracted out for additional work. Further, all specifications were created ahead of time because changes would cost extra. (Anyone who has done government contracting can talk to the shenanigans involved with it - I have not lived in this world for a long time.)

That said, we still do ad-hoc versions of many of these. For example, a system/segment specification today is an OpenAPI document between microservices. Most larger SaaS companies have the equivalent of a Software Configuration Management plan - Who can change terraform or a GHA, what are the standards that they conform to (linter, peer review standards).

[mpyne]

> This is a particular artifact of the government system process.

Yes, a government process meant to implement the waterfall approach.

If you look at Dr. Royce's paper which originated the concept, he was very explicit that it required upwards of thousands of pages of documentation to be written up front, if you were doing it "right".

By the time the required documentation had all been written, there should be essentially nothing left to do but to actually type out the punch cards as specified and turn then into a system of compiled programs.

Now, this appealed to government because it put documentation in place that was felt to be more viable for contracting processes, but ever since Dr. Brooks chaired a 1987 Defense Science Board study on the issues already facing the DoD trying to implement waterfall methods, they've been trying to restructure their software acquisition methods to pursue better outcomes rather than more concretely defined outputs.

Of course it's still a tremendous challenge for them even now, and it remains common to see defense acquisition projects that will say "Agile" to the right people even as they prescribe a full waterfall-style 'system engineering V' approach behind the scenes.

The ad-hoc responses that the commercial space often involves is usually more appropriate, believe it or not. They get process added when process is helpful, but not before it is helpful.

[ebiester]

I wrote about this - https://www.ebiester.com/agile/2023/04/22/what-agile-alterna... - Royce was describing what he saw as an anti-pattern that it was risky and invited failure without iterations.

(and my link to the Royce paper isn't working anymore - I need to fix that!) - I am planning on a followup that takes the last 3 years of change in mind.

[mpyne]

> I wrote about this - https://www.ebiester.com/agile/2023/04/22/what-agile-alterna... - Royce was describing what he saw as an anti-pattern that it was risky and invited failure without iterations.

Yes, that's why his paper essentially said "you're going to have to build two." One to figure out the mistakes you can't predict ahead of time, and the second for the real deal. Do your best to get through the first one as fast as you can, but still deliberate enough that there won't be any bugs left behind for the second one.

But a third or subsequent iteration was definitely a failure in his mind, and even building two (or one-and-a-half, depending on your framing) was simply a concession to the reality that actual implementation would run into unpredictable issues, for much the same reason computer science had already learned the halting problem was undecidable.

I have a book with his paper and to the extent he speaks of iteration as desirable, it is only iteration between succeeding steps of the overall 'waterfall'. E.g. in an ideal world you iterate between system requirements and their decomposition into software requirements (updating the system reqs as necessary to ensure the software reqs you're writing are accounted for). Likewise for system requirements to software analysis, and so on.

As you point out, he mentions that this concept is “risky and invites failure”, and goes on to allow for re-refinement and re-implementation of the software requirements and program design phases based on experience from the testing phase. But he goes on to emphasize: “However, I believe the illustrated approach [waterfall with reimplementation post-test] to be fundamentally sound”.

The rest of his paper then goes into the detail of these phases, and he specifically notes early on that there is a natural question, of how much documentation is enough? And he gives a very clear answer: “My own view is ‘quite a lot’; certainly more than most programmers, analysts or program designers are willing to do if left to their own devices.”

It's not an accident that the DoD software acquisition requirements based on waterfall as mentioned by the other comments were so numerous or onerous. As Dr. Royce puts it:

- “The first rule of managing software development is ruthless enforcement of documentation requirements”

- When asked to review software projects the first thing he does is review the documentation. If the documentation is seriously lacking his recommendation is to replace the whole project management and shift 100% of work to fixing documentation.

- “Management of software is simply impossible without a very high degree of documentation”

- If procuring a $5M hardware device he'd expect a 30 page spec to suffice. If procuring a $5M software system, he'd “... estimate a 1,500 page specification is about right.”

I wasn't pulling "thousands of pages" from thin air. It's right in his paper and he's extremely clear about this. It's not an off-hand remark, he goes on to justify why he thinks that mass of documentation is required.

I want to emphasize that he's writing from the problems he was facing in his era. Computer systems necessarily were room-sized installations, interactive computer time was incredibly expensive, but paper was cheap. There was no Internet to speak of to share powerful and efficient open-source libraries. There was no "continuous deployment" or "continuous integration".

The system had to work well pretty quickly after the subsystems were built, installed, integrated and tested or this newfangled computer system that cost millions in 1960s dollars to run per month would be nothing more than a money sink while the nerds tried to troubleshoot.

Nowadays we don't develop under those kinds of strictures and we've put tremendous investments into allowing real useful systems to be developed using the simpler processes that even back then were much easier to develop around, when it could be used (Dr. Royce's paper even leads off by describing the 'nice' process as he explains why you can't use it as system size grows). The voluminous test documentation he's propose are things we pretty much do write today, but we call them test suites and we grow them along with the program, rather than write them all months before coding.

I think there's a lot to be said for what a modern-day waterfall process might look like with the technologies and iteration speeds available to us now, the only problem is that I think it will still resemble agile more than it would resemble the process Dr. Royce described.

[rzzzt]

Indeed, I came across this not as a contractor but in my university textbook :) I wanted to collect the document list that forms the "thousands of pages" mentioned above in the waterfall model.

[mpyne]

Yeah and that's helpful too, because we typically talk about caricatures about both agile and waterfall and I think people truly don't realize that waterfall isn't simply "think about what you do before you do it" and nor is agile "code first; think later".

If people truly understood what waterfall is and how it's supposed to be carried out, they'd be less apt to recommend it. Nothing prevents a team from employing planning in an agile effort, but doing this doesn't turn it into a waterfall project and you shouldn't describe it as such.

If anything, teams that refuse to use agile (thinking it inherently means meetings, story points and not looking beyond 14 days) often send up choosing something even simpler, like cooking up a simple design doc of 4-6 pages before implementing it.

But that's still not waterfall, it's just another of the infinite renditions of agile methods that are out there, just without the consultancies issuing formal training certs.

[bdangubic]

at one point or another in my career (gov contracting) I had to write or co-write or review every one of these. and without fail, within 6-12 months they would be stale/inaccurate/obsolete/… the truth is, even on projects where sufficient time is allocated to write these, there is never (literally) time allocated to keep them up-to-date