[jerf]

Like many aphorisms, it's 95% true or so. Almost always when facing a social problem within a company you should be solving it socially. If you don't understand why that's true 95% of the time (i.e., trying to argue that it's completely untrue), you're not going to be able to identify the 5% of the time that it's false, and following the aphorism will likely serve you well. Understand the aphorisms before breaking them

However, there is that 5% that matters. In my opinion, one of the things that separates a good "architect" from "a really senior engineer with lots of responsibilities" is the ability to engage with the social and political implications of code design, and convert social concerns into lines of code. (This further reinforces the idea that architects must code, on the grounds that in general, this degree of understanding of a situation is not transferable. Especially to people who don't even believe these issues exist or are relevant to a design, which is quite a lot of even quite senior engineers.) The 5% is not equally distributed and a lot of it will concentrate on to the architects, so for instance I find myself thinking about this stuff a lot because of my position. But it's still the exception, it just happens that I get a lot of that exception.

Of that 5%, the vast majority of it relates to Conway's Law: "Organizations which design systems are constrained to produce designs which are copies of the communication structures of these organizations." I first encountered this in a list of funny jokes and such that had been knocking about since the old UNIX days, and at first I figured it was a joke too. Then as I grew more experienced, I thought it was a lament, in the style of "The first 90% takes 90% of the time. The remaining 10% takes 90% of the time." Now I understand it simply to be a law of software design. Like gravity, if you intend to defy it you'd better have a really solid story as to how you intend to avoid it. It is often much better to reorganize teams rather than fix this with code, but even optimal organizations of teams will still have a certain amount of Conway's Law issues, because the real world is messier than team structures should be. Plus engineers suggesting team reorganizations doesn't always go well; if nothing else you will confuse the heck out of your management even proposing it, like, "what are you doing thinking about that?". You may be forced as an architect to deal with suboptimal organizations anyhow.

And thus we wrap back around to microservices... once an organization hits a certain size, you are almost inevitably constrained to use microservices, because organizationally it is simply impossible to have 500 people working on "the same thing". You'll either use microservices, or a monolith that contains an ad-hoc, informally-specified, bug-ridden, slow implementation of microservices, even strictly de facto in the form of "this code belongs to this team, that code belongs to that team".

(The generic form of Greenspun's 10th rule is "Any sufficiently complicated program in a certain domain will inevitably be pushed towards certain solutions based on the domain, and if the designers attempt to resist that solution, that will simply result in an ad-hoc, informally-specified, bug-ridden, and slow implementation of those solutions." Or, in short, "use the right tool for the right job, even if you thing you don't like that tool.")

[marcosdumay]

> You'll either use microservices, or a monolith that contains an ad-hoc, informally-specified, bug-ridden, slow implementation of microservices, even strictly de facto in the form of "this code belongs to this team, that code belongs to that team".

That metaphor isn't good here. You gain nothing by including network communication between those different pieces of code. The ad-hoc solution is often more formally specified, less bug-ridden, and faster than the microservices solution.

Microservices solve some technical problems, not social ones.

[jerf]

"You gain nothing by including network communication between those different pieces of code."

I consider the network aspect of the communication a bit of a side show here. What's important is the information flow. Even if you've got a monolith, you've got some components, and if you are taking such good "advantage" of your monolith that the communication flow between modules is not something that is clearly specifiable and understandable because of the global variables and shared state and shared everything else, what you've got there is a mess, not a beautiful shining example of architecture that obviates the need for microservices.

Or possibly, you've just got something too small for the microservices to be relevant. Which is totally a thing, and I totally agree with the various things that say "Don't write your startup as microservices".

But eventually as your size grows large enough, you will either end up with components so nicely separated that putting a network between them doesn't affect them all that much, or you will have a big ball of mud that nobody can move around anymore. If you don't have one of those, you just aren't large enough yet. Humans can not code the way biology tends to work; we need bite-sized chunks of a larger system to get our heads around.

[marcosdumay]

Services interfaces are among the least maintainable kind of interface we have on IT, and REST services are at the less maintainable end of that.

When you add that network communication, you remove coherent type enforcement, timing restrictions, availability guarantees, causality guarantees, unicity guarantees, and likely more stuff that are just not on my mind right now. Those are all characteristics of the information flow.

[afriend4lyfe]

Conway's law has been popping up a lot lately for me. Can you share some typical examples of "communication structures of these organizations" and the systems they inevitably result in?

[jerf]

It's hard because in a small example you can always say "well why not just restructure the team some other way?", because there's no collection of real forces to balance against. But a simple example would be, let's say you're offering several services to the public. You want all of them to have a single sign on. Then you'd better create an "authentication" team, because if you leave authentication up to each of your 5 services, you'll end up with 5 authentication schemes. (Usually to make the "authentication" task large enough that team will also be responsible for the central record about billing info, contact info, etc.)

It's tempting if you don't believe in Conway's Law to say "What's so hard about having everyone share authentication services? We'll just mandate that they have to use one service without having to create a separate structural team to do it." But it will not generally work.

One element that contributes to this is that the teams are not just technical, they are embedded in a company in a larger context. If you create product teams A, B, and C, the larger company will be grading the management of A, B, and C on the revenue they generate, or other appropriate metric. Thus, the management for those projects will tend to be very focused on that metric. If you tell each of them that they are responsible for authentication, they will each want to spend as little as possible on it because it is not a revenue generator. (Who has ever signed up for a service because its authentication was just so awesome?) They will each correctly come to the conclusion that the cheapest thing is just to bash something that vaguely resembles authentication into their own system, because the costs of coordinating between the three teams will be too large. So you really have to carve out this interest into a team that will be measured on the goals relevant to authentication, such as scalability and security rather than revenue.

Engineers are also constitutionally inclined to say "Well, the solution is just to stop measuring teams A, B, and C by revenue and instead measure them by..." but then it's somewhat hard to figure out what the next words are. "Code quality" is something engineers would love to say, but even engineers on some level tend to recognize that does not actually put food on the table, so to speak. (It is an important element of considerations of how to spend resources, and I would agree that it is often underestimated. But it makes no sense for it to be the prime measurement of a team's output, even before we get into the issues involved in objective measurement of code quality.) It's really hard to avoid having at least some of your development teams being measured directly in dollars in one form or another, and from there the rest of these forces tend to flow.

[hinkley]

Would you agree that a project with less than twenty engineers that "has to" use micro services has issues completely unrelated to Conway's law?