[beat]

This is largely wrong.

No, people cannot know "everything in detail, in advance". That doesn't mean that they don't know anything. They know a lot. Nobody with any actual experience in requirements-gathering expects 100% perfection. So the underlying assumption about the underlying assumption is wrong.

After 20+ years in this industry, I'm long past believing the conventional wisdom that running systems are the best way to gather better requirements. It's not agile. Think about it. A key part of agile is to push everything to the left as much as possible - to catch problems as early as possible in the cycle. What's earlier than before you write the code at all? Writing code to find out what's wrong with it from a requirements perspective is really inefficient.

This isn't to say we shouldn't get working code out there as quickly as possible, or that feedback from working systems has no value. But this idea that it's the only way to get meaningful requirements, that's just BS.

Requirements aren't a document, or a process - they are a system.

[gilbetron]

The OPs statement isn't wrong, or even largely wrong, it is largely right. There was no statement of skipping all requirements gather, but skipping the idea that you can do one requirements gathering and have everything you need to develop the entire system.

Your push back to waterfall development is driving me crazy - we already tried that for decades and you can only get it to (kinda) work with a ridiculous investment that only makes sense for incredibly important systems, like launching a billion dollar rocket. And even then, you need iteration, just a more more careful, sandboxed type of iteration.

[beat]

So the OP was fighting a strawman. Like I said, nobody out in the real world believes in pure waterfall anymore. Everyone knows that, realistically, a completely up-front requirements process doesn't do enough.

But the quote agile unquote response is every bit as reactionary, and does happen out in the real world... "You guys start writing code, I'll go get the requirements". Writing code is expensive, even in agile process. Just because you're doing two week iterations or continuous delivery doesn't mean you no longer waste time and effort on dead ends. You're just dying by a thousand cuts.

Turning to user reactions to working code as the only requirements-gathering mechanism is stupid. Stupid. It ignores a ton of requirements issues that are not only complex, but dangerous to screw up - financial behavior, SOX and HIPAA compliance and other regulatory issues, and more. A mistake in initial implementation can cost millions of dollars, company reputation, and worse.

And again, what the OP is proposing here is not agile. Just because you're tossing code over the wall in short sprints doesn't mean you're agile. Agile means catching potential problems as early as possible in the process. Catching problems with requirements is almost always going to be cheaper than catching them by writing code and finding out that the code is wrong.

Agile requirements gathering is a thing, yo.

[dolessdrugs]

"nobody out in the real world believes in pure waterfall anymore"

I'd allow that this might be true within large software organizations, this is definitely not the case where most software is written: in non-software organizations.

[beat]

I'm reminded of something a certain high-end ops director (responsible for a DevOps push at a Fortune 50) would tell his CxOs... "No matter what business you think you're in, you're in IT now".

I work mostly in big enterprise companies. Whatever business they are in, they are "large software organizations", and they have decades of experience creating and evolving processes to suit the times and available. tech. You don't need to be Google to be an IT company. Any insurance company, any big-box retailer is an IT company. They know how to do this stuff, believe it or not.

footnote: Don't judge big enterprise companies by what they were doing 20, 30 years ago. They were state of the art then, and they're often state of the art now.

[dolessdrugs]

It's a question of support though, in a non-software-selling org, as a dev, you are a cost center, not a profit center, so getting the tools or other things you need is not a business priority; in fact, any additional costs in the cost centers are only losses on the balance sheet. In a company that sells software (primarily), you are the profit center, so anything that can be done to facilitate your work is supported, as it drives the bottom line.

footnote: just because they produce lots of software doesn't mean they've ever learned how to do it right. Ford is still a car company, Chase is still a financial company, Schlumberger is still an oilfield service company, despite all of them producing more software than some Software Companies.

[beat]

Do you actually work in these environments, or are you making assumptions?

Resource contention is a problem in pure software companies, too. I used to work for a small pure software company in rapid growth. What did we have? Legacy code nightmares that were as bad as or worse than anything I've seen in the Fortune 500 (like building the core product on antique Borland C++ where there were only 9 licenses in the company and new licenses were no longer for sale and hadn't been for years, while the UI was written in Java Swing with a table kit from an out-of-business vendor). And almost all growth money went to expanding sales staff... engineering got screwed. They sold (and sell) terrible quality software, and they make a fortune at it.

Meanwhile, I'm at a massive health care company, and they hired me because they're committed to radical improvement in how the already-okay software is built and deployed. We're working hard on a serious continuous integration pipeline, and I expect us to be as good as anyone in a year - our reference points for "Why can't we do this?" are companies like Netflix. We're after that level of smoothness in the process, and we'll get there, or at least get close.

Don't let conventional wisdom tell you who is and isn't good at software.

edit: I'm reminded of going to a meetup about selling to the enterprise in Silicon Valley some years ago, and the twenty-something Stanford crowd were convinced that because these big companies have big failures, that they must suck. I pointed out that if you worked at a startup with $50M revenue, they'd be pretty successful, right? I've worked on several projects with annual development budgets larger than that. It's expensive and risky because they're operating at scales that most of the HN crowd can't even comprehend.

[adrianN]

I've found that writing (pseudo-)code is absolutely necessary to find problems in the requirements. Often enough the requirements are self contradictory or just contain too many unnecessary corner cases. I've seen requirements that sounded really simple in the requirement doc, but turned out to be extremely hard to test because they implicitly defined a state machine with dozens of transitions.

[beat]

Yes, definitely. This applies a lot to infrastructure issues, too. But pseudocode or extremely simple test case code can do this a lot better than tossing something into production to find out if it sucks.

I suspect a lot of the HN hostility to proper requirements analysis is coming from writing trivial systems.

[carlmr]

Especially because English is often a terrible language to express requirements.

[adrianN]

Especially when it's written by people who aren't native speakers but work in a "we're a modern company now" environment.

[ako]

Core to agile is small incremental releases. Most technological innovation is done agile: in small releaseable increments. For example, We've been releasing small improvements for cars and planes for over 100 years. Every year a new model, with small improvements.

Humans are really bad at designing and building large improvements from paper requirements. Small improvements mean you understand most of the requirements are known and tested, and only small parts are uncertain.

The real problem is that testing requirements is really hard. You need to build the product to test the requirement. That's why most industries have an intermediate between requirements and product that is testable: this could be small scale prototypes, but more and more it's a virtual model that can be tested through software algorithms.

If we want to make real progress in the software industry, we need to move beyond word documents with requirements that are by definition not testable, to testable software models that don't require a full implementation. Low-code, model driven development is an example where this is happening.