[DanWaterworth]

The problem is that software has a tendency to become complex. The proposed solution is to break up the software into smaller programs.

There are certainly advantages to having smaller components. It allows you to rewrite components in a different language should you want to, for example. But there are disadvantages to: smaller components means dealing with failure at a much finer granularity.

In my opinion, the reason large programs become complicated is that there has been no emphasis on simplicity. Breaking components into smaller pieces forces you to adopt robust interfaces, but there are better ways of creating simpler programs.

My personal approach is to reason about parts of a program in terms of what they mean rather than what they do. I also have a strict rule that says, "don't change the meaning of a component, create a new one". This methodology works for me.

[dawkins]

Can you please elaborate on "reason about parts of a program in terms of what they mean rather than what they do"?

[DanWaterworth]

Of course.

A good example that many people are familiar with is parsing data. Compare these two approaches. You could write functions that manipulate the input and build up an output or you could create a structure that represents a grammar for the data you are parsing (perhaps by using a parser combinator library).

In the first approach, the only feasible way of reasoning about the program is operationally; when it gets to here, this function is called, causing ... . In the second approach, you can reason about the program by considering the grammar that you created. You don't need to know exactly how the parsing happens in order to understand a grammar. I argue that this is because the grammar has been given a meaning.

Parsing makes a good toy example, but this same technique of finding ways of giving meaning to components is applicable to software in the real world.

[jrajav]

> But there are disadvantages to: smaller components means dealing with failure at a much finer granularity.

I am really curious why you see this as a disadvantage. At my current job, I've had the experience of moving from a relatively small backend system that was broken up into discrete message-passing parts to a larger frontend system that was mostly one monolithic project. The former was by far much easier to debug, despite it being much older, less sophisticated, and having less logging, simply because it was easier to isolate and reproduce problems. The queues also made it very easy for us to bring down, diagnose, or scale individual components as they ran in production.

I couldn't see this as anything but an advantage, and one that's well worth the added complexity once you pass a size threshold.

And sure you can just go cowboy and say "I'll make it simple anyway!," but firstly, interface-enforced simplicity is not the only reason you would go with queues and RPC (other reasons have been mentioned in the article or by myself), and secondly, it is much more difficult than you think to enforce such abstract common disciplines on a large project.

[DanWaterworth]

I'm specifically talking about failure of components here rather than bugs. When you have more components that can fail independently, dealing with failure is necessarily more difficult, for the simple reason that there are more ways that you can fail.

> interface-enforced simplicity is not the only reason you would go with queues and RPC

No, certainly not. I shouldn't have implied that that was the case. However, if reliability, maintainability or simplicity is the reason why you are considering breaking up your large components then perhaps there is a better way of achieving your goal.

> it is much more difficult than you think to enforce such abstract common disciplines on a large project.

I'm speaking from my experience working on medium-large programs written exclusively by myself. I don't know that my methodology would scale to a team, but I also don't know that it or something similar to it couldn't.