[tsimionescu]

> Actually doing 'waterfall' properly would probably be fine? Or at least not the bogeyman it's made out to be.

Doing waterfall in the exact sense that is usually described will almost certainly never work well. People usually focus on the design phase, but I think the much more catastrophic part of 'waterfall' is doing testing only once development is complete.

Now, you absolutely can successfully run software projects that are design heavy, that more or less freeze requirements and designs early on, and that seek to execute on the design, instead of iterating. However, if you truly develop for 6 months before any kind of external QA, as described in most Agile talks about what Waterfall means, that is a recipe for disaster. If you do test things by feature, invest in component testing and practice that all along the way, you can succeed.

This essentially describes a 2-stage waterfall: design and execute, where the execute phase includes both development and testing all along the way. Is this 'doing waterfall properly'? Or is it a hybrid methodology? That's a matter of definitions in the end.

It's also important to note that the extent to which this works depends a lot on the project itself. If the requirements are volatile (e.g. when developing a new product with an uncertain market), or if the project is too large (e.g. if the design phase suggests it will take 5 years to finish given the current team), then it's likely that the project would benefit a lot more from an Agile style of development, where you deliver smaller chunks of the project to its end users to gather early feedback on the actual usefullness.

[jrumbut]

I think the parent post has, but for the reader who is a software engineer take 15 minutes out of your life to read WW Royce's "Managing the Development of Large Software Systems."

In this brief and engaging paper you will find the diagram used by many agile enthusiasts to describe the "waterfall method" and will be shocked to discover that it is held up as an example of a process that never actually works in reality.

You will then read quotes like this, which could have come out of an agile book:

"For some reason what a software design is going to do is subject to wide interpretation even after previous agreement. It is important to involve the customer in a formal way so that he has committed himself at earlier points before final delivery. To give the contractor free rein between requirement definition and operation is inviting trouble."

http://www-scf.usc.edu/~csci201/lectures/Lecture11/royce1970...

[qznc]

http://beza1e1.tuxen.de/waterfall.html

Incremental and iterative development is not an invention of Agile either. It was used before "Waterfall" existed.

[Frost1x]

I've used incremental and iterative processes in R&D for many years before "agile" became the current goto. Unfortunately, the obsession in tech with following trends has pushed many to try and modify, IMHO, a better refined process for the domain where agile just doesn't quite work. In R&D, prototyping work is quite challenging and doesn't lend itself well to full-out agile pipelines. CI/CD can actually hamstring you far more than help you, as can other typical PM tools. You need to be more agile than "agile."

For larger projects with adequate resources, agile can make sense, although it's typically modeled after quarterly focused business models and can miss long term opportunities. The core issue with agile is that its structure is ripe for abuse by everyone involved except the actual developer(s). Instead, developers have to reconcile all sorts of poor choices together in a fairly formal environment, leading to headache after headache.

[Jtsummers]

It's important to realize that Agile didn't invent anything. Agile was coined after years of work by numerous people to develop (or incorporate) different ideas into their methods. That is, as far as incremental & iterative development is concerned, history literate Agilers wouldn't claim to have invented it, but perhaps to have placed greater emphasis on it.

[jrumbut]

Indeed, that's some great added context to the linked paper and how the whole waterfall strawman was created.

[eru]

Yes, basically people never actually seem to be doing 'waterfall' or even aspiring to do so.

I do agree with tsimionescu that leaving testing (and debugging) for last invites trouble.

Btw, that's an excellent paper.

[NikolaNovak]

Agreed; all large, very traditional, very legacy waterfall projects I participated in had:

* Unit testing immediately upon developing any component.

* System & Integration testing starting at some point during development - overlapping phases

* User/Client Acceptance testing also overlapping with end of development (but with planned fix cycles)

* Performance testing... unfortunately depends heavily on the project and its perceived performance but usually in the latter third. Well managed projects see it as "Performance Testing & Tuning", and plan for several iterative cycles of testing and improving performance. Poorly managed projects see it as "Performance checklist" and have a single cycle with no time for tuning.

This is as old-school methodology as it gets, and yet it's been consistently refined and works well when managed well and in right situation/client. A lot of multi-year back-office ERP or infra work for large corporations or public sector inherently needs to adopt similar approach because it's just the nature of client environment. At highest of high levels it's about 5 gantt chart phases with heavy, heavy overlapping and lots of cycles for fixing/improving.

A two-stage dev + test without even an arrow from test to back... I have no idea how that'd work in either theory or practice. Waterfall doesn't have to ignore realities of the world... it just is suited to different realities than Agile.

[bladegash]

I think that you bring up a great point regarding a hybrid between agile and waterfall, within the context of government.

This is my own experience, but I was involved with a multi-year project for a federal agency that did exactly this. I think what many people don't understand is that the design phase is a byproduct of the internal processes necessary for even getting to the point of developing a system in the government. What I mean by this is, systems don't appear within based on good ideas or pitch decks, they start with policies, and based on those policies, procedures.

Why is that necessary in government? Because typically, there are laws, executive orders, regulations, and various other policy devices that have to be implemented at the agency level, there is a review period that allows for input from affected external agencies, etc. In some cases, there is even involvement from congressional committees and their staff members. This is especially true if appropriations are necessary for funding a program.

I can honestly say it took a year and a half just to develop the policies and procedures, to coordinate them, and to begin work on the system. That was ridiculously quick, given the scope of the project. The good thing about doing it this way, is you have a very clear roadmap at the outset. The downside is, the process for making significant changes to the system's functional requirements can be a challenge (e.g., changes to policy/procedures, another review period, etc.).

That being said, once we actually began the development phase we took a much more agile approach. We would hold daily stand-ups, regular testing/feedback sessions with customers, established product owner(s), etc. I would say it worked really well, but it was not without difficulties. Those difficulties are far bigger than agile vs. waterfall though, it's just the way the bureaucracy functions (for better or worse).