Ask HN: Are System Design documents out of fashion?

[throeawayDocs]

I’ve been hopping around a number of large non-FAANG enterprises for the past few years. In each, I keep observing a pattern - people seem uninterested in writing System Design / Architecture documentation.

I struggle to grok a system if I can’t read about the context. What are the goals/antigoals? What constraints was the system built under? What big architectural choices were evaluated? (e.g Serverless vs Containers)

Digging deeper I also see a lack of consideration for non-functional attributes like Performance, Capacity and High Availability.

Cloud paradigms can help to an extent with auto-scaling etc., but I feel there should still be a model for the expected Perf/Capacity, given the selected SW+HW stack.

In the case of High Availability, I continully see teams running critical Prod services without clearly documenting the expected behaviour if an Instance, AZ/Datacentre, Region or an external dependency goes down.

All of these observations have been in highly regulated industries with significant tech budgets.

So HN:

Do you teams document these architectural designs?

Have I perhaps lost touch with modern practices?

How do you grow teams (or defend the design from Mgmt / Auditors) if this information is held in peoples heads?

Appreciate your input and apologies as English is my second language.

[rjbwork]

I'm not asked to or rewarded for it. I don't even get performance reviews. In the past I've seen the effort I put into documenting and architecting rot swiftly when things are handed over to others. That said, I do document any manual processes. If there are very non-standard or complex things I will write something about it. If I'm doing tutorials or knowledge dissemination I'll do call recordings for later distribution. It just doesn't seem like I'm rewarded for that kind of stuff so it's not a priority for me.

Ultimately, I blame management and incentives. I think most line levels would like to spend a lot more time making things bullet proof, documented, commented, automated, and diagrammed. But management and leadership want features delivered and progress made, even at the cost of a less robust and understood system. This seems like a reasonable trade off, but it should be made consciously. I don't think that it usually is.

To answer your questions -

Do you teams document these architectural designs? Barebones wiki. Terraform.

Have I perhaps lost touch with modern practices? The tech industry is VERY heterogenous in terms of practice. It's not the 80's 90's with the regimented "this is how we build and deliver software projects" waterfallesque processes. A lot of people have been burned by that over the years and have decided to go to the other extreme and throw a lot of process out of the window. A lot are doing scrum, or kanban, or agile, or any of a dozen other names for how we actually get shit done.

How do you grow teams (or defend the design from Mgmt / Auditors) if this information is held in peoples heads? Tribal knowledge dissemination to new hires by various current employees.

[vchynarov]

I have seen no indication of system design documents falling out fashion. If anything, the problem is the initial, pure document is used as a reference for too long instead of proper up-to-date documentation.

I'm currently putting together a design document (very simple system and still high-level) to ensure my idea is communicated concisely and unambiguously.

The main motivation is I'll have a refined reference to point people to so I don't have to worry about repeating context to all possible stakeholders.

[wyaeld]

Good teams still document these. Amazon is apparently a leading example where the documents are considered the source of truth.

However many organisations have limited technical leadership enforcing quality, and management layers not willing to invest in these assets.

Unfortunately some 'modern' practices are being interpreted to make the processes like Agile the most important thing, and reducing the importance of things like code/system quality and knowledge management.

Having a good technical writer pays dividends

[austincheney]

Yes and no. I have seen major disparities on this matter within different large employers. It’s a matter of expectation differences attributed to populations.

Many times the factors involved are discriminatory starting at hiring time. Such factors may include graduate education, independent research/writing, invention or original solution creation. These people are more inclined to write and diagram, because they have prior practice doing so. The inclination to write is a behavior predicated upon that prior practice.

The larger population of software developers are not so inclined to write. There is no uniform foundation to define an acceptable baseline of qualification to practice writing software professionally. That means the distribution of competence is highly variable and not well measured. In case of poor structuring there is maximal opportunity for selection bias in candidate entry, which suggests preferential factors for candidate selection not aligned with performance (however that’s defined).

In economics this is called the blind leading the blind. Not only is the greater population not guaranteed to be competent but selection preferences reenforce that candidates are selected away from competence towards a medium distribution like that of a bell curve. If you are in this population, as most of us are, you must work harder to write and document anything only for much of it to be ignored.

[themodelplumber]

I've found it helpful to at least look at it as a personal preference, even if no one else wants it. If it helps you, do it.

Such best-practices are an interesting not-so-universal thing in a lot of areas. IMO this often has a lot to do with employer relationships.

I've found that a lot of people are really proprietary about documentation, (not even getting to architecture yet) and treat their time spent documenting as time spent giving away their negotiating leverage.

Sometimes this is a good point to consider though. The situation where you are happy with your work and the situation where the organization needs to start paying more attention to your value can be examined separately and you can make decisions about sharing your documentation work separate from decisions about the documenting itself.

Others don't feel a need to hold back, but they would rather work on more apparent or pressing issues (i.e. more obvious to them because of their differing work perspective). It is then left to the contingency thinker to justify themselves, which can be really difficult.

Regardless, it's important to decide how you feel about it and how helpful it is to your perception of your value and efficacy, as well as the outcomes you desire in your relationship with the organization. Good luck.

[0x008]

> I've found that a lot of people are really proprietary about documentation, (not even getting to architecture yet) and treat their time spent documenting as time spent giving away their negotiating leverage.

I have observed this also. I’ve even had colleagues voicing this explicitly to me. However, I think it is a little bit near sighted. A good manager will never assume documentation can replace a senior developer who helped build the system from the ground up. However, a bad manager might make this mistake.

But in reality I feel like there are two types of companies. The ones that understand that the experienced developers are their true capital and so everything to keep those with experience - and the ones who live under the illusion that they can substitute individual people with processes and organizational structure. Those will also let good people go instead of paying them what they are worth, because of “compliance” reasons or company policies and will pay a multitude of what the extra salary was in lost time and burned hours afterwards when the people who come after try to understand the ins and outs of the legacy system.

[throeawayDocs]

This is an interesting perspective that I hadn’t considered.

My colleagues all seem happy in their roles, although they work in countries with less workers rights/securities(?) than in my own country. Maybe they don’t document anything so they are seen as “sticky”.

However I do wonder how they would feel if they were blocked from taking a new internal role because they’re deemed too essential, because they’ve not documented anything.

[themodelplumber]

Yep, it's tricky...It could also be that there's a need for clarity, for example a minimum objective level of architecture documentation and test-passing... It might help take the pressure off of individuals who have a high personal standard of work, with little perceived recompense from the org.

[twunde]

As others have mentioned design docs/BRDs, etc generally fell by the wayside with the rise of agile, especially in smaller companies and startups (colleagues from those environments may never have seen a real design document and possibly never had to document anything). However, over the last few years, I've seen many companies moving towards at least light-weight design documentation typically using ADRs https://adr.github.io/ and/or RFCs (https://github.com/rust-lang/rfcs). This is often because without these types of documents, its harder to grow teams as new members must either be taught be a senior member or learn from the codebase, both of which are slow.

As a secondary note, regulated companies may have architecture/network diagrams and disaster recovery plans but these may be located with the compliance documents and not in the general document store that engineers use.

[throwaway24124]

At my current company we write a proposal for any new project/service (outlining 3 different paths for building a new service, or the option to "do nothing"). Then, once an async discussion happens on the google doc between the team and stakeholders, we settle on moving forward with one of the proposals (or "doing nothing"), and write a 10-20 page design doc outlining the entire service, architecture, etc. Then another round of async feedback between stakeholders on the proposal. Edits to the design doc are made based on feedback, and the design doc is then available for reference for anyone.

I really love this process, as a new member of the team I can go back and read the design doc for any of our systems and understand not only the architecture, but also why certain decisions were made. The caveat is that this a remote-first company with a strong asynchronous culture, which forces a lot of these discussions to take place in a google doc vs a conference room with no record.

[enterdev]

What is your role/job title?

[throwaway24124]

Software engineer

[throwaway81523]

Not documenting is one of those things like open-plan offices, that management thinks will save money but that makes things worse in practice. In the old days programmers hated documenting. Now, the programmers (sometimes) want to document but management decides on slamming out more features instead.

[hyperman1]

I simply try to document the as is architecture, and found out I can't really do it as I want.

As I see it, I have a lot of tiny items like servers, clusters, applications, database schemas, services, network things, ... all interlinked by relations. Relations are two way, and typically of the kind 'contains' or 'talks to'. The main problem: There are a lot of them, even if they are all tiny.

I'd love to have a good way to organize all this. wikis require too much clicking around. I'd love to ask questions like: What goes down if this subsystem disappears. So anyone willing to share experience is appreciated.

Now all this is simply about documenting what is. The AskHN wants to go a step further: How to document what should be.

[bjourne]

Design documents are artifacts for PHBs to collect. They don't work in real life. What works in real life are guides and how tos. You want the software to do X? Here is a GUIDE on how to do that. You're curious about this library's API? Here are EXAMPLES on how to use it. These documents needs dates so if the how to was last updated in 2014 you know it's probably out of date so you don't bother.

[vanusa]

Yup, way out of style. Because it's, you know "waterfall" (as in, anti-Agile). Can't have that.

And because, well, gosh -- "reading". Especially all those icky complete sentences and stuff. Which has definitely gone out of style, especially among the smartphone-addicted set ("if I can't digest or respond to it while thumbing my phone, it doesn't really exist").

Have I perhaps lost touch with modern practices?

No, they've lost touch with you.

[throeawayDocs]

Appreciate your input. Perhaps it is seen as a waterfall way of doing a things. Personally I would write it alongside building the system, then continue to evolve it as the architecture matures. It does not need to be a case of writing everything down first before building things. Although it might be cheaper to change the words in the document compared to destroying Infra and rebuilding it!

This can then be used to show customers or auditors that we have been diligent and designed the system carefully, even if they find some flaws.

[vanusa]

I'm a huge fan of specifications (despite the downvotes to my comment above).

They can be slightly tricky to maintain and to get right. But not nearly as ticky as maintaining software for which there's at best only a partial sense of a coherent shared mental model available to the stakeholders (not to mention the developers) as to what this piece of software is supposed to do. Other than a slew of JIRA tickets written by a cast of thousands, and assigned to a zillion different developers over the years.

[faangiq]

It’s a race to the bottom. Buy the ticket, take the ride.

[nextUpFANG]

At Meta, we also use them widely. They're critical to understanding large systems and how non-functional features are affected.

[enterdev]

Do you guys work off a standard template?

[throeawayDocs]

Have you seen any open sourced standard templates? Would you be interested in collaborating on one?

[derwiki]

It’s not strictly enforced but encouraged in my current org. To be honest I do it because ultimately it saves me time though, so I would probably still do it even if it wasn’t encouraged. (Large “fangish” company)

[captaincaveman]

I think in general (sounds like FAANG do, but devs from there tend to bounce around FAANG companies), unintentionally, we have created a generation that don't know how to write a design doc.

[awa]

At Google, design documents are a big deal and I have really come to appreciate them a lot. My previous employers were not so big on design documents so it was a big change for me.