[sam0x17]

There are also a lot of cases where people "modularize" code without actually modularizing it --- they extract certain functions into separate modules or files just to break up the current file, but that new module they've created can't function or do anything on its own outside of the context it was extracted from. So in these cases they've really obfuscated the code in the name of "modularization", but the code is no more modular than it was before -- it's just more obfuscated.

[lostcolony]

Right; breaking it up doesn't necessarily make it more modular, it just necessarily spreads it around. This is -a bad thing-, with no other context. The hope is that it modularizes the code enough to enable better understanding/reuse/extension (thus being a necessary evil).

[josephg]

Right. The problem I've seen way too much is modularizing code too early in its lifecycle. This mistake seems to happen a lot by smart programmers who are inexperienced.

The instinct seems to be that they want hinges in their code, so their code can adapt and be reusable between projects. But they don't actually know where the hinges should go, because they don't need them yet. So they just put hinges all over the place - even where hinges aren't useful. If the metaphor is confusing, I'm talking about things like making an interface around a class, when there's only one implementor of that interface anyway. Or breaking a complex function into small, "reusable" pieces spread over multiple files - except where those small pieces are only ever used by that one call stack anyway. (And where they aren't that semantically self contained.) The result is harder to follow code with no actual benefit. And the resulting code is almost always bigger and more complex, and thus harder to work with.

Usually code thats the easiest to refactor is code thats the easiest to understand. That means, small, dense, correct code, with a simple test suite. If you write code knowing that you can (and will refactor) later anyway, the result is almost always better software. You will come up with a better design after you know more about your problem domain. Plan for that, and set yourself up to take advantage of that knowledge later.

[deltasixeight]

> Right. The problem I've seen way too much is modularizing code too early in its lifecycle. This mistake seems to happen a lot by smart programmers who are inexperienced.

I don't see this as bad. Modularization protects against an uncertain future. Most code that's modularized is only used once and this Looks bad only because you haven't seen the alternative of what could have happened if code wasn't modularized.

Non modularized code is often designed wrong because of an uncertain future. Once the project is too far down the line, people just keep piling technical debt on top of the design flaw. Code is rarely rewritten until it's at a point where it's horrible than it takes a massive engineering effort to rewrite and even this rewrite could be wrong.

The alternative is code littered with modules that are used once. Which is better? Obviously the more modular code.

[josephg]

> Code is rarely rewritten until it's at a point where it's horrible than it takes a massive engineering effort to rewrite and even this rewrite could be wrong.

This is your problem, right here. Your instincts know when refractors and rewrites are appropriate. But if you live in a corporate culture where those things are never allowed to happen, of course you’re going to run into problems. Premature modularity is a poor man’s substitute for simple-then-refactor. It’s not a good process.

[deltasixeight]

>This is your problem, right here.

Nah. There's no problem here. This is the most common behavior in any place corporate or not. Humans resist change. However specifically for corporations, it is very very very rare for a company to allow a rewrite because of two reasons: First the company is often way to busy with creating features and solving problems then to do a code rewrite. Second it is in direct conflict with the bottom line. Business people don't see the necessity so there is huge resistance.

When a company allows a rewrite it's 99% of the time only to serve a new feature or fix a flaw that no longer can be ignored.

>Premature modularity is a poor man’s substitute for simple-then-refactor.

Do you have any evidence to back up your claim or is this just your opinion? Using words like "poor mans substitute" doesn't lend any credibility to your claim. For example I can say the opposite and we can go in circles forever: Writing unmodular code is a garbage technique only done by junior developers who can't abstract things and by a good number of senior developers who've never learned how to code properly in a modular way. These guys don't even understand the true meaning of a module.

See what I did there?

[josephg]

> There's no problem here. This is the most common behavior in any place corporate or not.

Its hard to say statements like that universally because it really depends on where you work (and have worked). There are plenty of places which make time to refactor along the way, in small and large ways. Its good engineering, and a healthy code base nets better long term outcomes for the business too. Business people care about the team's velocity (since programmers are expensive). And I'm disheartened you think its the norm but not all companies have MBAs telling software engineers how to do our jobs. (Eg, Google / Facebook / Netflix / Github / etc.)

> Do you have any evidence to back up your claim or is this just your opinion?

Its just my professional opinion, backed up by almost 30 years writing software at all sorts of companies and in all sorts of environments. But no, I wish we had quantitative evidence for this stuff. But there's a weird dearth of quantitative research around software engineering. We as an industry don't really know what we're doing yet. I'd love to read some studies on this stuff if you can find any.

But I'm very confident about what works best for me, from trying lots of things on solo projects. I'm way more productive when I throw something together quickly, iterate like mad on how it works internally, then tidy it up and modularize as I get more confident around the best way to structure the code. Whenever I modularize up front I basically always end up regretting my abstraction boundaries - and either have to rework them (which is expensive) or I leave them alone, and just ship vaguely mediocre code.

But if prematurely modularizing your code is the only way you can modularize anything at your workplace then - well, you do you. But I find that pretty sad.

[deltasixeight]

If you can move your code and spread it around it is Modular by definition. This is never a bad thing from a design perspective. It is only a bad thing from a complexity and readability perspective.

More likely you think it's a bad thing because your code isn't actually modular. Likely you need one piece of logic but that logic isn't modular so to move it to another location you need to drag a bunch of extra baggage around with it. You wanted a banana but instead you got the gorilla holding the banana and the entire jungle. Sound familiar?

The smallest primitive that is modular is a pure immutable function. If the modules you are moving around are not pure functions then likely your code isn't actually modular.

[sam0x17]

> If you can move your code and spread it around it is Modular by definition.

For it to be truly modular, you also need to be able to use it in multiple contexts. I could take any random 5 lines of a complex function and pull it out into another function in another file, but that doesn't guarantee that this was a smart thing to do in that particular scenario. What I'm saying is there are tons of times when people do this merely to get the linter to pass, instead of for the actual purpose of modularization.

[Jtsummers]

Right.

If A and B are coupled so tightly that one cannot exist without the other, you don't really have two modules. You have one, AB. This is a common problem with improper application of the idea of modularity as well as OO concepts (in particular, breaking things into classes and thinking it creates a module). The latter is particularly pernicious in languages which don't have a clear separation between classes and modules (Java, for instance, or historically I think this has changed).

The reality is that there are two (at least) kinds of modularity. "Syntactic" (I need a better name for this) modularity like Java classes, or C's translation units (why I need a better name). These act as modules, but don't necessarily define a real, proper module. And then there are your real modules which are comprised of the things that must exist together (like in the earlier example), regardless of the project structure or language's notion of a module.

I had a team try to convince me their code was "modular" but their GUI portion was directly tied to the DB portion and other logic. Nothing could be instantiated separately, so it wasn't really modular, it was just using the language and build system's notions of modules to create the illusion of modularity (C++ and VS in their case). In contrast, another team had developed a collection of libraries (C# and VS again) that were used in multiple applications. That was real modularity.

[josephg]

Yes this. With I often think about questions like:

- Is this code useful in other contexts? Are there other contexts where I might want this? (Or someone else might want this?)

- Modules have a clear API boundary

- Modules can be described independently of the code which uses them

- Modules can (and should) have their own testing suite, independent of the test suite of the containing module

There's lots of examples; but you can kind of spot code like this in your projects. It has a property of being disentangled. "This just solves this one specific problem I have with strings / event emitters / random numbers / my database, separate from anything else I'm trying to do". "I kinda want to just document this code separate from the documentation of everything else". "I don't want to pull that whole library in, but can I just steal these 3 functions for this other project?"

[deltasixeight]

> This is a common problem with improper application of the idea of modularity as well as OO concepts (in particular, breaking things into classes and thinking it creates a module).

Exactly! The class is not the smallest unit of modularity. You unionize several pieces of mutable state with several functions (aka methods) in class based programming and if your unionization was a design flaw you're pretty much screwed. To make your code truly modular you need to break it down further. Separate state and function. To go even deeper make the function pure and make state immutable. Abstract mutability to a small unsafe portion of your code.

>The reality is that there are two (at least) kinds of modularity. "Syntactic" (I need a better name for this) modularity like Java classes, or C's translation units (why I need a better name). These act as modules, but don't necessarily define a real, proper module. And then there are your real modules which are comprised of the things that must exist together (like in the earlier example), regardless of the project structure or language's notion of a module.

The only true logic module is a stateless function that is independent of all context. Think on that, that is literally the smallest unit of logic that can be moved around anywhere.

The problem here is how do you program in a way such that even for these functions are easily de-composable and rearranged without necessitating a rewrite in the case where you find the logic needs massive changes?

Use point free programming with pure non-procedural functions solves this. The problem is mutability must happen somewhere and usually this is abstracted to a very small section of your code.

>I had a team try to convince me their code was "modular" but their GUI portion was directly tied to the DB portion and other logic. Nothing could be instantiated separately, so it wasn't really modular, it was just using the language and build system's notions of modules to create the illusion of modularity (C++ and VS in their case). In contrast, another team had developed a collection of libraries (C# and VS again) that were used in multiple applications. That was real modularity.

GUI programming is inherently hard as the state of the GUI is by nature muteable and it's hard to write modular code as a result. However modern programming frameworks generally get around this using the technique I outlined above, see React redux and MVU. https://guide.elm-lang.org/architecture/

[deltasixeight]

Read the 2nd paragraph of my post you replied to. You talk about true modularity and it is addressed in my second paragraph. If you find these problems with your code then the code was Never modular in the first place.

Modular code involves writing code independent of context. One way of doing this is to wrap every expression in a pure functional context. Another way is to make every variable immutable.

> I could take any random 5 lines of a complex function and pull it out into another function in another file, but that doesn't guarantee that this was a smart thing to do in that particular scenario.

Doing this doesn't hurt. There's nothing stupid about it unless you coded everything in a way where it's NOT modular. Procedural programming is usually not modular because the results of a computation depend on the Order of the procedures. Immutable state is isomorphic to an expression so procedural code with immutable state solves the issue.

Let's say I need a function to add 3 numbers. I impliment it like this:

  addThree = func(x, y, z){ return addTwo(x, y) + z}
  addTwo = func(x, y) {return x + y}

You're saying addTwo is unecessary and pointless if it's not reused. I'm saying you can't predict the future, there may be a time where you need addTwo, but if you never need it, it doesn't really matter, you don't lose anything here. Modularity doesn't hurt the structure and organization of the code. It only effects qualitative aspects like how easy is it to interpret understand or read.

[lostcolony]

"it doesn't really matter, you don't lose anything here"

Except you do. It's harder to understand and less readable; and in a real life rather than made up example, addTwo is in some other module entirely and has side effects that are completely invisible (in most languages) when just looking at addThree, making debugging and understanding WTF is going on far harder.

You mention "pure functional context", and yet this whole thread exists under a post about OOP, which is all about state management; it's no good trying to create toy examples that are side effect free to try and dismiss the points being made. Yes, of course pure code is fairly trivial to slice and dice, and the cost of doing so tends to be low (not zero, but low), but that's not really what is being talked about.

Otherwise you're kind of "no true Scotsman"ing this; "if you find these problems with your code than the code was never modular in the first place" - yeah, that's the point. Splitting code doesn't by definition make it more modular.

[deltasixeight]

>Except you do. It's harder to understand and less readable; and in a real life rather than made up example, addTwo is in some other module entirely and has side effects that are completely invisible (in most languages) when just looking at addThree, making debugging and understanding WTF is going on far harder.

You don't lose anything from a hard structural standpoint. Readability and being harder to understand is an opinionated based metric. It can be influenced by how you even name a function. It is a soft metric therefore weaker than the hard one. Additionally I DO mention the qualitative opinionated cost of modularity several times in this thread. So I address it but it is definitively weaker because anyone can have an opposite opinion and say that "in my opinion modular code is more readable"

>You mention "pure functional context", and yet this whole thread exists under a post about OOP,

Did you read the article? The author of the article was not convinced by OOP, he never supported OOP and generally the entire article is critical of OOP. I would say my reply is highly relevant and that you didn't read the article.

>it's no good trying to create toy examples that are side effect free to try and dismiss the points being made.

The examples serve a purpose to illustrate my point not to illustrate a real world example. If you want a real world example look at the react + redux architecture. The gold standard model for all web UIs exactly do what I mention with state management. They completely separate State away from pure functions. The entire web generally follows this model abstracting state management away from even code. Web apps are stateless with the database handling the entire job of state management. Examples of this pattern are EVERYWHERE, the example I gave is a toy example to help you understand not to say "hey this is a real world application." I thought that was obvious, my fault for not being clear on that.

>yeah, that's the point. Splitting code doesn't by definition make it more modular.

Completely and utterly wrong. Splitting code by definition makes it more modular. See: https://en.wikipedia.org/wiki/Modularity

The definition from above says: "Broadly speaking, modularity is the degree to which a system's components may be separated and recombined, often with the benefit of flexibility and variety in use"

If you are "splitting" code then by definition you are separating code which was previously combined. By Definition it is more modular. You really can't argue this point. More likely what's going on here is you have your own personal fuzzy definition of modularity which you are unconciously combining with your opinionated view on what constitutes good code. I am using the english definition not some fuzzy notion of good code so I am definitively right.

Be more exact and formal with your terminology it will lead to much less misunderstandings.

[sam0x17]

> Doing this doesn't hurt. There's nothing stupid about it unless you coded everything in a way where it's NOT modular.

It does. If I open up an arbitrary Ruby on Rails app for example, it is going to be easier to navigate for example a super fat model file than it is to navigate one that has been divided up into 10s of helpers and companion modules. There are similar scenarios in every other language. Extracting code and moving it somewhere else can truly be a death by a thousand cuts. My rule of thumb is if there aren't at least two places in the code that need to call My Extracted Thing (tm), then it shouldn't be extracted in the first place. If you're being DRY, it's OK to have a long class/file/method/function/module, and it's probably preferable to obfuscating your codebase and making it more difficult to navigate.

[deltasixeight]

>Extracting code and moving it somewhere else can truly be a death by a thousand cuts.

This is not what I mean by modularity. Let me be more specific. The code CAN be seperated and it can be recombined without a structural code rewrite. That is what I mean by modularity. Actually dividing code up and organizing it among different files is an opinionated organization scheme and NOT what I'm talking about.

When you do extract code and move it anywhere it's not exactly "death" it's just harder to read. Structurally the code is still sound. You can meet the requirements without rewriting the code, it's just the code is more complex to understand.

The bad scenario I'm talking about here is when you can't extract code and you can't move it around. When your requirements necessitates logic to be moved around and you can't because code is too coupled with other code. That is more closer to literal death by one cut. You are technically unable to meet requirements with existing code.

The modular primitive that prevents this from happening is the immutable pure function. But it's not a one size fits all solution.

[deltasixeight]

I would say the file thing is unnecessary. That's just an OCD thing. You can break stuff up and keep it within the same file, it literally has the same effect.

I understand why you think that type of modularity is bad, but it is actually good. It only appears bad because most of the "modularized" code is only used once.

No one can predict the future so the way to minimize rewrites is to make composable units of code that are small and highly modular. It's not about breaking up your code. It's about writing small logical component then building up the larger component by composing the smaller components.

This results in a large number of modules that are only used one time, but it prepares your code for the inevitable point of the future where you find out the design was wrong and you have to rearrange the logic.

When such a time comes you most likely just trivially rearrange some of your logic and add additional pure functions into your pipe line for any actual new logic. The majority of your modules remain untouched and this only seems bad, but it prevented a rewrite of the entire framework.

The timeline where the definitively worse outcome occured didn't happen because your code was too modular. So you have no point of comparison and you assume the modular code that is hard to read is bad only because it's hard to read. You failed to see how it prevented a massive refactor.

Usually if code is so unmodular, people just live with it and keep accumulating massive tech debt on top of everything. If all your seeing is tiny functions everywhere then this is definitively better than the alternative.

From a design perspective highly modular code is always better. From readability perspective though, you are right, modular code is harder to read. But there are ways to mitigate this.