[jacquesm]

You have rediscovered the job of Software Analyst, which until the early 90's was a thing. Then that all got upended and we ended up with a mix between product owners, project managers and developers / devops but I think that that ignores the fact that Analyst is a different set of skills.

[ebiester]

There is a lot of room to reevaluate the lessons of software development pre-web in the context of the current environment.

Like, if waterfall of a project can be done in 2 weeks, is it agile now?

[mpyne]

> Like, if waterfall of a project can be done in 2 weeks, is it agile now?

Sure. The thing is, the waterfall guys would tell you it's impossible to do it in 2 weeks because you need to have written down everything first. "Thousands of pages" was the terms they used.

Agile guys would point you to the Agile manifesto which would lead you to "working code over documentation" and "people over process".

A 2 week period to go from initial spec to product in a user's hands to capture feedback and make changes from there is much closer to agile than to waterfall. In fact it's more or less exactly some older versions of Scrum (which didn't permit deviating from the planned sprint user stories midway through the sprint, instead changes influenced the subsequent sprint).

[rzzzt]

The DoD's 2167 standard from the late '80s mentions the following documentation that should be produced as part of the development process (section 6.2 and Appendix D):

- System/Segment Specification

- Software Development Plan

- Software Configuration Management Plan

- Software Quality Evaluation Plan

- Software Requirements Specification

- Interface Requirements Specification

- Software Standards and Procedures Manual

- Software Top Level Design Document

- Software Detailed Design Document

- Interface Design Document

- Data Base Design Document

- Software Product Specification

- Version Description Document

- Software Test Plan

- Software Test Description

- Software Test Procedure

- Software Test Report

- Computer Sytem Operator's Manual

- Software User's Manual

- Computer System Diagnostic Manual

- Software Programmer's Manual

- Firmware Support Manual

- Operational Concept Document

- Computer Resources Integrated Support Document

- Configuration Management Plan

- Engineering Change Proposal

- Specification Change Notice

[ebiester]

This is a particular artifact of the government system process. These are contracted pieces of work that Company A would deliver, Company B would administer, and Company C would be contracted out for additional work. Further, all specifications were created ahead of time because changes would cost extra. (Anyone who has done government contracting can talk to the shenanigans involved with it - I have not lived in this world for a long time.)

That said, we still do ad-hoc versions of many of these. For example, a system/segment specification today is an OpenAPI document between microservices. Most larger SaaS companies have the equivalent of a Software Configuration Management plan - Who can change terraform or a GHA, what are the standards that they conform to (linter, peer review standards).

[mpyne]

> This is a particular artifact of the government system process.

Yes, a government process meant to implement the waterfall approach.

If you look at Dr. Royce's paper which originated the concept, he was very explicit that it required upwards of thousands of pages of documentation to be written up front, if you were doing it "right".

By the time the required documentation had all been written, there should be essentially nothing left to do but to actually type out the punch cards as specified and turn then into a system of compiled programs.

Now, this appealed to government because it put documentation in place that was felt to be more viable for contracting processes, but ever since Dr. Brooks chaired a 1987 Defense Science Board study on the issues already facing the DoD trying to implement waterfall methods, they've been trying to restructure their software acquisition methods to pursue better outcomes rather than more concretely defined outputs.

Of course it's still a tremendous challenge for them even now, and it remains common to see defense acquisition projects that will say "Agile" to the right people even as they prescribe a full waterfall-style 'system engineering V' approach behind the scenes.

The ad-hoc responses that the commercial space often involves is usually more appropriate, believe it or not. They get process added when process is helpful, but not before it is helpful.

[ebiester]

I wrote about this - https://www.ebiester.com/agile/2023/04/22/what-agile-alterna... - Royce was describing what he saw as an anti-pattern that it was risky and invited failure without iterations.

(and my link to the Royce paper isn't working anymore - I need to fix that!) - I am planning on a followup that takes the last 3 years of change in mind.

[rzzzt]

Indeed, I came across this not as a contractor but in my university textbook :) I wanted to collect the document list that forms the "thousands of pages" mentioned above in the waterfall model.

[bdangubic]

at one point or another in my career (gov contracting) I had to write or co-write or review every one of these. and without fail, within 6-12 months they would be stale/inaccurate/obsolete/… the truth is, even on projects where sufficient time is allocated to write these, there is never (literally) time allocated to keep them up-to-date

[jacquesm]

That doesn't do justice to either waterfall or agile.

[ebiester]

Oh certainly - I'm conflating the adjective of agile with the manifesto of agile. I've been on projects with multi-hundred page design docs and multi-week UATs. And nobody wants to go back to prince2 for example.

The point I was trying to make is we should be diving back into the older methodologies and accumulated wisdom and re-evaluate some of the older dead ends with new context.

[user3939382]

Or referring more to the process of building the specs, requirements engineer(ing). Imho agile became a way of hand waving most of this critical process and responsibility, in place of a new inefficient and ill-defined process.

Yes, you don’t know the nuances of all specs upfront and revision will be necessary. Turning the ship with arbitrary degrees of freedom outside of bullet points on a roadmap is not an efficient way to resolve that for many projects.

[moffers]

I came up as a software requirements analyst before the weird transition between business analyst to product owner to product manager to technical product manager. But living in requirements for 15+ years really gave me a leg up on these “let’s go back to requirements!” efforts.

[jacquesm]

It always amazes me how bad the software world is at keeping lessons learned as learned, especially when compared to say engineering. It's as if every 20 years or so we throw away the books and reinvent it all from first principles, hopefully this time with fewer mistakes overall but usually we end up finding both new ones and re-do some old ones.

[enraged_camel]

It's because software engineering, which deals with bits, evolves dramatically faster than other engineering disciplines, which deal with the physical world.

[arvid-lind]

It's almost like "move fast and break things" isn't such a good mantra.

[physicles]

It’s almost as if we need to write some unit tests. For our profession.