[quotemstr]

The most important lesson for people in power in big tech companies: FORMAL PROCESS IS THE ENEMY. Don't let the accumulation of gatekeepers and approvers and other capital-P Process let things become 100x harder than they need to be.

Do not confuse difficulty with prudence. Your organization will become slow and ossified by default unless you take specific steps to maintain flexibility. One of these steps is to install a culture of change. Things that are easily undone should not require approval to be done in the first place.

Do not allow long-timers in your organization use "caution" or "good engineering practice" as an excuse to slow everyone else down. Emphasize that the most important part of software development is moving fast. Let new people try new things. Minimize the number of people who can say "no".

Most of all, do not just _believe_ people who talk about best practices and software "quality" and stuff like that. Most of the time, they're just finding fancy-sounding ways of saying "nothing should change unless I say so".

[erikpukinskis]

I just learned about Hegel so now everything is a nail, but...

You really can't make categorical statements like "formal process is the enemy". You put formal process into place, and that creates a dialectic where you're waiting for someone to come in and champion the antithesis: "formal process is the enemy". Then that works for a bit until it doesn't and someone synthesizes: "sometimes formal is better sometimes informal is better". Then that becomes the thesis.

Then there's a new dialectic in place. Someone will come in and say "the character of our company is such that we're generally better with formal processes". Then someone synthesizes that, and suggests that all processes are on a trajectory from formal to informal and the company is good at expediting that.

Then someone realizes we need a process for regular deformalization.

This continues forever. None of these ideas are really that much more generically true than any other.

So to embrace Hegel, the goal is just to question yourself as far as you reasonably can, so as to get the best view on the current situation and only the current situation. Make a choice and move on, knowing the next situation is likely to be different.

Edit: I should add this is the process by which we hurry into The Future, which some people might point out means turning our backs on Everything That Is Holy and is therefore a tool of Satan, but I'll leave you to explore that with the rapidly growing universe of anti-Hegelian YouTubers.

[freekh]

I really liked this comment :) I am looking forward to pondering upon this way of reasoning for this entire Saturday :)

[kbos87]

I have a hunch that you've never worked at a growing mid-sized or large company that is clinging to its startup roots and is averse to any sort of process at all.

Being unwilling to adopt a necessary amount of structure and process is just as much of a hindrance once you hit a certain size.

That may be a newer or less common problem, but it's a very real one.

[el_benhameen]

Agreed. I've often found that those who are resistant to any amount of (reasonable) formal process are those (usually long timers) who were most empowered by the implicit process and rules. In the end, they're also saying "nothing should change unless I say so".

[35bge57dtjku]

This and the parent comment describe exactly what happened at the company I just left. I wonder if the newly hired TPM will ever have any real power there.

[beachy]

Very insightful.

[linker3000]

I can totally identify with this. I joined a company that had grown from 2-3 people in a shed to 40-50, providing global B2C, B2B and company Intranet design and development (A Drupal house). I was tasked with providing some structure and process to (initially) the support team in order for the company to be in a position to cope with expanding workload and look 'corporate' to chase larger accounts. When I joined, the company was pretty much firefighting all the time and diverting people from project to project according to which customer shouted the loudest at the time.

Every change I proposed was questioned to death and generally dismissed, I was micro-managed within an inch of my life, and I found the two key Directors just could not let-go of any control of their 'baby' (the company), to a point where any tasks I tried to assign to team members were met with pushback because the Directors had already assigned them other things, told them to work on other projects or asked them to report back before accepting any work from me.

One time when I was asked to work on a proposal to take over the migration, management and support of the Intranet for a large local University, I completed the initial RFP document and reported back to the Directors with a plan for a meetup with the potential customer because some key points needed elaboration, and some of the SLA terms could not be met without additional team resource, but they told me they had already phoned the Uni and agreed to go ahead 'as is'.

I left after 9 months.

[banned1]

"Emphasize that the most important part of software development is moving fast."

No, that is not the case most times. The most important part of software development is, as with any other part of the company, to maximize the value of the company. That may be speed, but if you have 1m users the most important thing may be to preserve trust, to acquire new customers, to automate existing infrastructure, etc. etc. Speed of software development is not a goal in many environments. Thoughtful execution may be hindered by a cowboy attitude in development. I'm not saying that speed in development is not sometimes the goal (maybe in startups within an enterprise trying to test an idea), but often it's not the most important thing.

[inopinatus]

"Maximising company value" isn't a useful end goal you can give to anyone but the CxO team. For everyone else, it's a motherhood statement. Trotting it out as though it's some kind of engineering compass isn't helpful.

To this you've appended the hackneyed conflation between "speed" and "cowboy attitude". It's a common category error, but in truth, rapid iteration is one of the enablers of high quality and is a huge effectiveness multiplier for top shelf dev teams. Slow development merely enables lazy programmers (since it conceals their apathy) and gives us waterfall disasters. "Thoughtful execution" is possible in any context, and most thought processes are greatly enhanced by evidential techniques such as rapid prototyping / feature spiking.

Fast+incompetent = cowboy attitude, sure, but the problem there isn't the fast bit.

[geofft]

I've found maximizing company value to be a pretty good end goal as an individual contributor (and it's definitely the thing I'm most excited about doing when I walk in the door in the morning). It's how you tell whether "fix this bug" or "implement this feature" is higher priority, or whether "do this thing now in our current system" or "experiment with this completely different system" is higher priority. You can do all of those at speed; the question is which to do. And getting it wrong dooms your company, because you're wasting effort.

In a well-functioning company, I don't need to make that decision myself, of course: I have management who has evaluated the cost of having the bug and the benefit of having the feature, or who's experienced and smart enough to determine whether doing the short-term thing now or investing in the longer-term thing will be likely to have better payoff. Because this process is there, if I trust my management, I don't have to spend-time second-guessing them (or worse, doing the research myself), and then I get to move quickly. But again, moving quickly isn't the point. It's a tool on the way to delivering value.

[quotemstr]

> Fast+incompetent = cowboy attitude, sure, but the problem there isn't the fast bit.

Exactly right. Frequently, in industry, I see people confuse the "slow" and "competent" bits. They think that just because a given change needs to get a bunch of sign-offs that the overall effect of the system requiring the sign-offs is a net benefit to the organization.

It's not. It's paralysis, and you won't notice how ungainly and slow you've become until a different org a tenth of your size manages to clone your entire feature set in a hundredth of the time it took you to develop it, all because your developers have to deal with bullshit all day long and your competitor's can just act.

A lot of people will claim that you need gatekeeping and approvals and such to maintain a rigorous standard of quality. You have to be brave enough to disregard their advice. You have to be able to take the gut-wrenching step of saying, "No. We're not doing that. I don't care if this change caused a SEV, we're not adding process. Moving fast is important!".

It's very easy to believe the nostrum of "no pain, no gain". But sometimes you have to realize that pain is just pain.

[sillysaurus3]

You have to be able to take the very demanding step of saying, "No. We're not doing that. I don't care if this change caused a SEV, we're not adding process. Moving fast is important!".

The moment that some minor disaster hits, which it inevitably will, whoever is in charge will find themselves no longer in charge.

I think this is one of the fatal flaws of a big company. Everyone is terrified of risk. I'm not sure it's possible to counteract this tendency unless the company was built from the ground up not to punish risky behavior (e.g. Facebook, supposedly).

[quotemstr]

Right. You have to create a culture where it's okay to fail. And it's not enough to just say that: you have to actually practice it. You have to let people make mistakes, then do nothing so that other people don't feel afraid to take risks. Hell, reward people for failing. Highlight them as exemplars of people who do things. If you don't fail once in a while, you're not moving fast enough.

[logicfiction]

Rewarding failure can have pitfalls (if we mean something like customer impacting fallout). You may then be encouraging people that create broken things over people that more quietly create well-functioning things the first time around.

Enabling people willing to act is important though and I fully support the do nothing aspect of failure, or better yet support and stabilize but not reward.

[sillysaurus3]

Yeah, but this is at odds with virtually every big company. No one has the power to affect that kind of change except the CEO, and the CEO is rarely interested or involved enough to do that. Is there any way around this problem?

[tonyedgecombe]

I've seen urgency used as an excuse for sloppy work so often that I can't ever imagine the speed of development being a good metric.

[inopinatus]

I don't conflate the two. Urgency and speed are two very different metrics. Many lumbering waterfall projects have moments of urgency or even panic.

[sanityUnbounded]

I recommend Creativity, Inc. to every person in a management position whom I have the opportunity to recommend it to. Although Et Catmull presents a depiction of Pixar and it's growth as a largely artistic endeavor, I feel that it is applicable to those that manage engineers especially.

The ratio of students that decided to major in engineering because they were good at math in high school vs. the students who decided to major in engineering because they they wanted to make something inspiring, I feel ( and let me stress my inexperience as a current undergrad, without experience in the actual field ) is vastly favored in career growth towards those that decide to create instead of simply learn.

While there is always room and lucrativity to tie engineers on a leash and juice mathematics and logic out of them until they reach an existential crisis, there is always (ALWAYS) room within the budget to let them set out upon a path which is in tune with their not necessarily young, but foolish and hopeful beliefs.

[ryandrake]

I didn't take that away from the article at all. This wasn't an argument against formal process. It was more a story about acting like you're a big company, while pretending you're a small company, and the pitfalls of doing this. Where "acting" encompasses pay, stock incentives, other kinds of talent motivation, recruiting, leadership style, innovation, and yes formal process.

There's nothing wrong with process and best practices. Process is often scar tissue left behind after critical mistakes in the past. A company without some formal process is a company young enough to not have made disastrous mistakes yet.

I've worked in a company of more than 50 that was basically without a trace of software development process. It was a clown circus. Nothing could be released on time reliably, quality was a joke, there was no documentation, no code review, and no plan or roadmap. Sure, they moved fast, but it didn't matter.

[geofft]

> Emphasize that the most important part of software development is moving fast.

This doesn't seem to me to be true. Can you expand on your thoughts here?

I'd agree that the ability to move fast is an essential part of effective software development, but I don't think it's the most important (the most important is producing business value accurately and reliably), and I think that moving fast for the sake of moving fast is likely to result in moving quickly in the wrong direction. In particular, it's easy to be so worried about moving fast that you have no time to take new business needs/opportunities into account (see also Tom DeMarco's Slack), and it's easy to get excited about shiny new technology and spend a lot of time implementing it and roleplaying Google for no reason, which is a great way to move very quickly while not delivering a cent of business value.

Finding somewhere with formal process, with full-time managers who aren't reluctantly moved to management for career advancement, with clear hierarchy and goals, was an important part of my most recent job search, simply because, as a good engineer, I hate to see my work go to waste. I have heard the same sentiment from lots of my talented friends.

[quotemstr]

Unless you take conscious steps to remain flexible and to eliminate barriers to productivity, you're going to find your ability to "produc[e] business value" diminished because effort will get sucked into process instead, and for reasons nobody can identify. Soon enough, you'll find yourself wondering why you have so many developers, all of whom appear to be doing work, yet without getting anything done.

By default, all software development organizations slow down. I say that developer velocity is the most important goal because, without focusing on this velocity, you end up solidifying, like a cooling lava, and it becomes difficult to accomplish any of your other goals.

[geofft]

Well, okay, you and I are agreeing that velocity is nice to have, but why is it the most important goal?

I can make the exact same argument about automated regression tests. As the size of your codebase grows and the number of customers (i.e., people triggering edge cases) grows, unless you take conscious steps to make sure that every build doesn't regress previous bugs, all your effort will get sucked into diagnosing and fixing things that you've fixed before. And all your developers will be fixing bugs but you won't be fixing new bugs and certainly not shipping features.

In particular, if you want to refactor / rewrite parts of your code to improve development velocity, you'll suffer from the well-documented problem (see e.g. https://www.joelonsoftware.com/2000/04/06/things-you-should-... ) of losing all the knowledge of the weird edge cases you've fixed along the way. So, testing is strictly more important than development velocity!

I can make the same argument about a half-dozen other development best practices, and I'll have actual citations to back them up. Why is development speed the most important goal, why is formal process the enemy of speed (instead of its friend, because it makes sure work doesn't get duplicated or wasted!), and what is your evidence for these claims?

In particular, if your claim is true, we would not expect to see as much feature development / new products from old, large companies (I'm thinking of Oracle, Microsoft, Apple, Amazon, etc.) as we do. We would expect to see these companies grind to a halt and perform poorly in the markets (investors really want to see growth, not just continuing to be as good as you were last year), and that's not what's happening.

[dasil003]

Your focus on "process" is missing the deeper truth: companies don't slow down because some busybody managers put in processes, it's because as organizations (and codebases) grow, they become more complex and more difficult to change.

It's not a question of preventing formal processes, doing that indiscriminately is a recipe for disaster; every engineer will do things their own way, and once the head count exceeds the ability for everyone to talk to each other daily, there will be an explosion in complexity and duplicated effort. Rather, you need to put in the right processes, maintain them just as you would code, and kill them when they no longer serve the company. Where "process" goes wrong is when it's used as a hammer to address issues that would be better served by team structure, common tooling or architecture. For instance, if you have a quality issue, adding more code reviews will generally cost a lot more, and do a worse job than making the responsible team wear the pager for their own code.

The real secret sauce to staying agile as a company grows is small, cross-functional teams. This is where SOA (microservices for the kiddies) really pays for its overhead—by decoupling human teams and giving them agency. This won't make a big company as agile as a startup, because that's literally impossible, but Amazon and Facebook have shown how much more agile it can make you than any prior megacorp.

[geofft]

> This is where SOA (microservices for the kiddies) really pays for its overhead—by decoupling human teams and giving them agency.

Funny you mention microservices, because I've been thinking about Susan Fowler's book about them and how the advice is basically "Here's how we made sure that we kept up our agility by introducing tons of process before anyone could think of deploying a microservice, because otherwise every team would do literally everything their own way and it would be great until it turned into an unquenchable tirefire."

I'm not aware of Uber's technology organization having stopped producing business value. That company has a lot of problems, but the inability of their software engineers to engineer software doesn't seem to be one of them.

[rleigh]

Used appropriately, a formal process provides structure to the workflows within an organisation. Everyone knows the process, and it provides a framework and expectations for everyone to work within. In organisations without any process, it can lead to difficulty in getting stuff done because there's no responsibility or accountability for who is doing what, and when. Who is reviewing my PR, which CI systems tested and deployed the prospective change, why was the work done, when does it get merged, and by whom? Etc. Basic daily routine.

You also have the situation that people invent rules and requirements in the absence of a documented process, which can change arbitrarily and are impossible to work with because it depends upon the changing whims of whoever feels they are in charge. Clairvoyancy and divination should not be a job requirement!

Process becomes problematic when the process grows and ossifies and becomes the end itself, rather than the means to an end. You end up with it being impossible to get work done because the process is so burdensome. When your bosses care more about the process being followed to the letter rather than the actual completion of the work which the process should be enabling, that's IMO the point where it needs a rethink. Unfortunately, organisations have a tendency to accrete the stuff; I've in the past argued for less to the amazement and incredulity of my bosses. It's also a factor that process is used as a blunt instrument to wield power over others, and I think that's certainly a driver as well. But often organisations favour accountability and rules over creativity and efficiency, even if you spend 95% of your time with process, and 5% actually doing productive stuff...

[jjtheblunt]

That's exactly why I left Apple engineering, and I've not regretted it, somewhat sadly, somewhat happily.

[dclowd9901]

Risk is the enemy of establishment and profit. You _will not_ convince a powerful train to change direction simply because there's no tracks ahead.