Hacker News new | ask | show | jobs
by sbellware 901 days ago
Some additional context on this:

- There are additional organizational and process mechanics in-play that are out-of-scope for just a write-up on commit messages

- The team is made up of around 15 people supporting around 400 repos and about 8 live products

- The team DOES NOT use pull requests. Contributors are expected to be socially capable and fully engaged in the inter-personal, realtime communication that supports high-performance, low-handoff work.

- The ticketing system is NOT the primary means of communication between team members. It's merely a catalog of work items with only high-level supporting notes. A work item, as the old saying goes, is a "placeholder for a conversation", rather than a work order that can be executed upon by a contributor operating in isolation.

- The original author is describing an exceptional software product organization. He's not debating the merits of various subjective perspectives on what kind of personal flair might drive the content of commit messages. Instead, he's describing one small piece of a methodology that explicitly discourages personal flair and explicitly encourages (and provides supports for) direct communication, and rigorous attention to the team's well-considered, proven, and communicated norms.

- Contributors on the team are expected to maintain their own work logs in the spirit of engineer's logs from engineering industries. If the "why" of a change is important, it will be recorded in the work log. Senior technical leaders (at a minimum) keep up with the individual work logs looking for hazards and maintaining an understanding of the learning that's taking place in the team.

- Contributors on the team are expected to maintain equipment logs for the components that they're making changes to. This isn't the same thing as a SCM log, as an SCM log is for the SCM, not the product under development.

- The reason that the "subject-first" approach is used is to get contributors to talk specifically to the impacts made to the software itself, rather than talk about themselves and their labors. It's about training developers (and designers, and operators, etc) to be more objective about the work they do, and to communicate it in an objective way. Mentions of "changed", "fixed", etc, are records from the perspective of the worker, not the product. When scanning the list of changes made to a work product, the team members are primarily interested in the impacts to that piece of equipment first and foremost. The work product change logs are voiced in terms of the work product, and not the worker. Again, the reflections, learning, questions, observations, etc, of the humans involved in the work go into the human's own logs, and are supported by the person-to-person processes that are beyond the scope of the article on commit messages.

- These 15 people with their 400 repos and 8 live products was winning industry awards within a couple months of go-live within the first year of the team's charter. There's nothing average about the team, the product, and the methods in-play. The commit messages are just one detail of the finely-honed process and culture that were intentionally designed, socialized, and painstakingly-supported to achieve its objectives.

- The author of the article is describing one facet of an overall product development system for a high-performance software organization. He's not trying to spark a dialog about the myriad opinions of how commit messages are written across the breadth of software development within processes and cultures that may not go much beyond the cross-referencing of items between ticketing systems with an SCM system, which is arguably a higher-ceremony approach from an artifacts perspective, but also a "bare minimum" from a process and culture perspective.

It's not entirely pertinent, but just in case there's some question as to whether this work is being done in a business of some scale or other at some particular time or other in a business's lifecycle that accounts for how the development system works, and whether it's a special case: the business context of team's work is a $1B multi-national with about 2,000 employees working in the regulatory side of corporate law, finance, and venture capital.

The business context isn't the enabling factor or the deciding factor. The product development system in-effect is the key factor, and it was shaped explicitly for its outcomes irrespective of the business context. The development system has been used at various companies up-and-down the axis of business scale and scope.

The single greatest contributing factor of the team and its work is that they carry very little technical debt, and the methodology required to work this way touches every molecule of process, organization, and culture. It would not be recognizable from the perspective of typical mid-curve software development, and it would not be understood by examining any single process element in isolation.

2 comments

> The team DOES NOT use pull requests. Contributors are expected to be socially capable and fully engaged in the inter-personal, realtime communication that supports high-performance, low-handoff work.

I am interested to know more about your workflow given that you do not use pull requests, could you please elaborate on that?

I cannot imagine that you’re all just pushing to master.

sbellware's reply says just about everything of import, but from a tactical perspective, yes, we do push to master. Sometimes we push regular commits and sometimes (if we branched) we push merge commits. We have hundreds of repos and it's very rare for there to be conflicts of note. We do reviewing as we work via pairing (and on demand, live, when necessary).
I've always found it odd (at least since 2001-ish) that developers see the 1-work-for-1-person as some kind of desirable norm, rather than some arbitrary fixation that has been burned into them since the start of their careers.

It's wishful thinking at best to expect effectiveness and productivity come from the isolation of developers to their own isolated work items. It's hopes and dreams of promoted-beyond-their-competency managers to parallelize the workforce to maximum capacity, rather than to focus on the ability to make progress without slowing down, and then optimize from that baseline.

It's odd to me when someone has to call out "pairing", rather than just assume that work items are taken on by work cells and work groups, and that the more important objective is continuity of knowledge and understanding amongst the workers so that momentum can always be transferred immediately without having to reset and relearn.

The 1-work-for-1-person ethos is so obviously counterproductive that I struggle to understand how it's still so prevalent. Except that human history is rife with popular mythology, and humanity's efforts have been undermined by it for as long as there have been humans.

It speaks to the dark ages period of software that we live in that even the production systems in software work are so antiquated that they still try to grasp at the most implausible straws, not to mention ones that have already been debunked and disproven by industry at large.

It speaks to the reality that software development largely still operates as a craft rather than an industry. Software developers barely pay attention to industry as a body of knowledge, and the sorry state of both software products and software jobs reflects it.

When the day comes when "pairing" never even has to be stated, software work will have entered its modern age of industry. It should be presumed that the processes of momentum transfer between knowledge workers is the singular element that dictates whether a team can continue to build momentum in perpetuity, or sputters out prematurely early in a project's lifetime.

We always have to mention "pairing" when talking to people from that dark ages, and those people will never have a good frame of reference for it due to seeing it through the lens of a more primitive grasp of industry and the body of knowledge that surrounds it. It's almost impossible to explain a high-functioning production system to anyone who has made no time in their professional lives to become as qualified in industrial theory, organizational theory, and production systems as they have to learn the latest convoluted front-end toolkit.

We're like cavemen working with tools gifted to us by some advanced civilization. We've been trained to use high-tech, but we barely understand the systems of work that produce it, and we don't allow ourselves to benefit from the things we can learn from more advanced and more mature fields.

But then, as long as the software tooling available to us is a bad is the things that we make with it, we'll never have the buffer of spare time needed to make the study. And down and down we go.

Pull requests are a relatively recent addition to software development. What did you do before pull requests? If you haven't been around for that long, and have had a career that always had pull requests, then pull requests seem like an inevitable and irreducible inherent feature of software work. But they're not inherent to software work.

I'm going to generalize "pull request" into other terminology so that I can describe the deeper issue:

A pull request is an asynchronous handoff, and asynchronous handoffs are a known sub-optimizer of production processes of all kinds, not just software. It causes batching and queueing, and the harmful effects of those on processes are also well-understood.

Why would I want to introduce asynchronous handoffs into a process that we've worked so hard to optimize at every level and in every turn?

Our process is what it was before GitHub popularized pull requests, and popularized workflows that increased the engagement with their digital product.

The problem is that developers aren't asking whether pull requests are even a good thing to build a development process around. Given the detrimental effects of instituting asynchrony as a desirable target, I find surprising that software workers have gone all-in on something that is an obvious production system anti-pattern.

But to a degree, I'm also not surprised. There's a prominence of a certain kind of psychology in software development that wants to work in isolation without having to be bothered by things like realtime communication and coordination. It's seen as a kind of privilege in that cohort of software developers, but it's one of the most harmful things that can be done to software work (or any work where more than one person is involved).

Software developers tend to be antisocial, and there are enough antisocial developers gathered in this industry that they can enable and reinforce the idea that being antisocial is desirable.

We have a zero-tolerance posture when it comes to antisocial tendencies amongst developers. Our processes are all realtime and synchronous. The level of productivity that we're after requires zero-tolerance posture with regards to batching and queueing and handoffs. We collapse the whole rotten mess of software development down to the fundamentals of production systems and reap many multiples of productivity relative to industry norms specifically by harnessing the momentum that comes from always moving forward, rather than contending with the pervasive start-stop interrupted processes that are common in software development due to failure to recognize the effects of batch handoffs.

The specific details of the work flows aren't as important to disregarding pull requests as the character and behavior of the people in the process. We're not antisocial, and we don't accept antisocial behavior in the team. We work very closely with each other in tightly coordinated and highly-communicative ways in everything we do. At any given time, a very small number of active work items are being worked by a single developer.

In the end, we know more about production systems itself as a field of study and body of knowledge than the average team of devs with 5-to-10-years of experience. And we know a lot about structural design. We marry these two concerns into a process that would be quite recognizable to someone who's studies Lean and the work methods that Edwards Deming championed that changed the face of human productivity decades ago - in every field of human endeavor EXCEPT software development.

So, for a team that is irretrievably mired in asynchrony, and operating on a belief system that has never questioned the workflow fundamentals and the detrimental effect that asynchrony has on the upper potential of productivity, I can see the utility of pull requests. But it's not something that I'd willfully volunteer for. I won't trade the luxuries of working in asynchronously isolation for the vastly-improved lifestyle of rejecting the myth that technical debt is inevitable. This is a lesson that American production industries learned when Japanese producers started beating western producers at their own game by exploiting the virtuous relationship of quality and productivity.

The lesson is that productivity and quality are indistinguishable at the the upper limits of high-performance work and work teams. The level of quality that can generate multiples of end-to-end productivity of an entire process and production system doesn't survive in the presence of a tolerance of asynchronous handoffs.

We don't use pull requests because we want to have a more higher quality of software development life, and we can't have that if we indulge our endemic sense of entitlement to working in the dark all by ourselves.

There's an old saying that was oft-repeated in the days before Scrum executed its ethnic cleansing campaign on XP: If code reviews are good, do them constantly and continuously.

We don't do pull requests because every moment of our work is a code review simultaneously with code production. When you learn how to put together an entire production system with these fundamentals as the non-negotiable parts of the work, the productivity that will emerge will be revolutionary.

Most software teams can't imagine how good and how easy software work can be when the target is the upper echelons of quality.

That said, software work has been removed from this body of knowledge for so long (largely due to the rapid increase in the software development labor pool that started around 2010) that software development culture at present at large no longer has possesses the knowledge of what mistakes are in software work.

Until software developers gain a fluency in software structural design, no amount of fiddling with team workflows will result in anything my negligible, middling improvements.

Thank-you for the insight. The related articles would be worth further reading.

> development system has been used at various companies

Is there a name for it?

What resources could point us to the idea of "software as engineering?"

The words "equipment log" and "work product" sound pretty unique. Maybe older books address this.

Are Erl's SOA books and Yourdon's on OOP still relevant?

Erl and Yourdon's books are definitely still relevant, but they have to be put into historical context and historical perspective at this point in order to extract the enduring value that they have.

I wouldn't literally build a SOA system the way that it was done back in the mid-2000s, so I know what to disregard in a legacy SOA book, and what to give attention to. Same for an OO book from the 90s. But without having that sense of history, and a fluency in fundamentals and principles, the necessary background that puts these books into context and makes them useful today would simply be missing.

It's not uncommon for developers to pick up a "classic" software book and come away with all of the wrong ideas.

sbellware may have more to add, but there isn't a name for it. We focus more on the "it" than the branding of it. We do use the word "continuity" to describe our goal: https://github.com/aaronjensen/software-development/blob/mas...

> The words "equipment log" and "work product" sound pretty unique.

We call them "material logs", as equipment logs are more or less a subset of those. Just think about the sheet in every bathroom at a store that says when it was last cleaned, or the clipboard attached to the factory machine that lists its maintenance record and problems. Or the patient record outside of the patient's door.

"work product" just means, the product of our work. Nothing special about that one, I don't think.

> Are Erl's SOA books and Yourdon's on OOP still relevant?

I haven't read either, personally, though I have Yourdon's book en route. We use the term "structural design" quite a bit, so I'm curious to see Yourdon's take on what they call "structured design". From everything I've read about it, it sounds very similar to a lot of how we think about things as it seems to be the basis for much of the coupling and cohesion thought in our industry.

I'd also recommend studying Lean (not Lean Startup, which has much to do with Lean as non-fat yogurt does).

Right. We don't name the methodology. If we did, people who claim to be "doing it" by mentioning the name of the thing. It's the individual practices that are talked about, and that's on-purpose. We can't afford to have the system of practices be obscured by the kind of grasping at status that can come from being able to claim something by name. It's a means of protecting the methodology itself from decay, and from protecting the people on the team from the shallowness that inevitably making something "a thing".