[qznc]

Does anybody know some monorepo horror stories?

I have heard plenty of people complain about their many-repo structure and wishing for a monorepo. I would like to hear some concrete story where a monorepo went wrong. This article is just abstract opinion.

[mr_tristan]

Oh yes. #1 problem I've experienced at multiple places: delaying integration with tons of branching.

Usually the PM or PO force everyones to using some vague "product version" they track for public releases. Places that use monorepos well tend to have very few branches (like, maybe 2 or 3), which have nothing to do with your public product versioning scheme, and instead use "branch by abstraction" to stay integrated.

But what I end up seeing, is that the product team and middle management gets involved with dictating version control, e.g., "this will be version 1.2, and then that should be 2.3, ok let's cut those branches...", and then they change their minds as some team has to delay and before another is ready. And then bugs start rolling in from both testing and they don't know what to do, and they start asking people to just "get it done", and then, things really start falling apart. You add 10+ teams trying to use branches for their own work based off of god knows what and it becomes a mess of crazy integration problems.

I seriously think that a huge benefit of multiple repositories, is that is scares the pseudo-technical managers and product people into not bothering with trying to track or dictate usage of the version control system.

[beirut_bootleg]

My experience is similar. The newly promoted CTO used a botched merge and delayed release to justify the move to a complex but half-assed monorepo branching strategy with minimal tooling and documentation, trashing any systems teams had in place. We've started botching releases, losing prospective clients, and now are in release limbo.

[the_gipsy]

Or you could just tell them to stay off versioning/release/branching discussions, even in a monorepo. All they need to know is what release/deploy has what features, and when the release happens.

[mr_tristan]

Easy to say when you're starting from scratch, but in all of my cases, these were "pre-existing conditions", i.e., was in place before I joined, and it took a long time to get people to see the light. It gets real tricky when you're _not_ a manager, and you're basically asking other managers to stop having such a loud voice in things.

Just another case of "culture eats strategy for breakfast". Once people start using any particular versioning strategy, it becomes canon, even if it's a terrible way to organize.

[the_gipsy]

Yeah sorry if I came off as dismissive. It's usually a long process, if that's how things run.

[flukus]

Not horror stories, but I've used mono repos in a company with 50+ projects and the results tend to be tight coupling, macro level spaghetti and libraries being left to atrophy due to fear of change.

For example I can be working on project B and need to make a change to Lib A, so I make the change commit my work and now project Z broke. Now I have to learn whatever the hell project Z is because it's not my responsibility and we may not even have anyone responsible for it. Then I have to work out of the changes to lib A need to be reverted, backward compatible or if project Z needs to be updated. This sort of thing with 10 libraries and 40 apps and the complexity that every individual developer has to deal with goes off the charts.

Separate repos with versioned packages don't necessarily fix this but they do let you manage it a lot better, whoever is working on project Z can update it's version of lib A at an appropriate time (or never).

[Too]

Sounds like a feature and not a bug. It prevents irresponsible breaking changes of lib A interface and just hoping that some other Z team will clean up after you.

Postponing the required Z change to later could be seen as beneficial in some scenarios but what if the change you made to lib A was a security fix, then you would want all apps of that lib to be forced updated right away. Then your your change should be backwards compatible, monorepo or not.

If you want to have reusable components then make sure they are reusable, if you want a special version of lib A that only works with lib B you are essentially forking lib A making it not longer a reusable lib, just a subdir for project B. Interface versioning could help with such non backwards compatible changes, in a monorepo you normally do this with a /2.0-directory.

[flukus]

> It prevents irresponsible breaking changes of lib A interface

But sometimes breaking changes are necessary and a mono repo provides no responsible option way to make them. We keep things backward compatible where possible but this creates just as many problems as it solves when cruft builds up over years.

> just hoping that some other Z team will clean up after you.

Ideally you notify them and they'd keep somewhat up to date, but the yes it's up to the team that work on that project to do so. I can't be an expert on everything but the mono repo assumes I am.

> in a monorepo you normally do this with a /2.0-directory.

Now you've reinvented version control and package management.

[dwohnitmok]

Not sure if it's a horror story, but I was on a team considering breaking up a monorepo mainly due to build times. Basically our CI would build from scratch a lot more often than we'd like rather than incrementally. It'd also run the full suite of tests more often than we would like. This made builds often take something like half an hour, which if there were multiple PRs open could force a wait of an hour or more before an approved PR could be merged (since CI would need to re-build and re-test a PR if another one was merged before it). This was exacerbated by a version of the branching problem mentioned in a sibling comment.

This wasn't an insurmountable problem. Something like Bazel probably would've done wonders here rather than our homegrown incremental testing logic (as well as nailing down why incremental builds weren't happening). Personally that's where I would've invested time rather than splitting up the repo. I'm not sure what ended up happening. I moved onto another project before seeing the conclusion of that conversation.

FWIW, a younger me led a break-up of another monorepo into a multi-repo that I now regret and think caused more pain than it was worth (likely because I split the repo along the wrong lines). And so I disagree with the premise of the article. If you split your repos incorrectly, you can cause more pain than not splitting your repos at all. Long build times are annoying and a velocity-killer. Moreover in the long run you can get ball-of-mud problems that repo boundaries make harder (that was probably the biggest impetus for why I wanted to break up the repo in the first place). However, incorrect version linking due to miscoordination of fast-moving dependencies in different repositories is a production-services-killer, and that caused us no end of frustrations. This was in addition to the annoyances around the fact that we had several different JVM languages that all had different build systems in each repo meaning that cross-repo edits were even more difficult than usual to corral together locally on a developer's machine since the build artifacts all depended on each other, but this was expressed in different ways in different repos.

Just as a bad abstraction is worse than no abstraction, I believe bad modularity is worse than no modularity.

Note that tooling helps this as well; tooling that exposes the transitive dependency chain of production services can reveal inconsistencies in what you thought was the version of a dependency that was deployed and what was ultimately deployed. But that means that both multirepos and monorepos need tooling.

[aidenn0]

I asked a similar question a previous time the mono- vs many- repo question came out, and the few responses I got were roughly "The repo became many tens of gigabytes which was unwieldy"

[andrewprock]

Unfortunately, breaking a 10 gig repo into 5 repos often means you have to now download five 2 gig repos.

[detaro]

That can still be better, e.g. when part of the problem are tools that work on a per-repo level and fall over/get slow if the repo gets to large.

[andrewprock]

How do these tools work when it comes to working on multiple repos? I imagine they either don't, or that it's worse than working on a single monorepo of the same size.