[draw_down]

Right, dev boxes do not need to do double duty as a personal computer plus development target, which allows them to more closely resemble the machine your code will actually run on. They also can be replaced easily, which can be helpful if you ever suspect something is wrong with the box itself - if the new one acts the same way, it wasn't the dev box.

I don't recall latency being a big problem in practice. In an organization like this, it's best to keep branches up to date with respect to master anyway, so the diffs from switching between branches should be small. There was a lot of work done to make all this quite performant and nice to use. The slowest part was always CI.

[tmpz22]

I feel like we're not getting the right lessons from this. It feels like we're focusing on HOW we can do something versus pausing for a brief moment to consider if we SHOULD in the first place.

To me the root issue is the complexity of production environments has expanded to the point of impacting complexity in developer environments just to deploy or test - this is in conjunction with expanding complexity of developer environments just to develop - i.e. web pack.

For very large well resourced organizations like Stripe that actually operate at scale that complexity may very well be unavoidable. But most organizations are not Stripe. They should consider decreasing complexity instead of investing in complex tooling to wrangle it.

I'd go as far as to suggest both monorepos and dev-boxes are complex toolchains that many organizations should consider avoiding.

[jrochkind1]

> I'd go as far as to suggest both monorepos and dev-boxes are complex toolchains that many organizations should consider avoiding.

I'm not sure "monorepo" means the same thing to you as it does to me? To me, it just means "keep all the code in one repo, instead of trying to split things up into different repos."

To me, it's the thing that is the simple solution, it just means "a repo" -- the reason it gets a name is because it's unusual for large orgs with enormous codebases to have everything in one repo, it's unusual for them to do the simple thing that works fine for a small org with a normal codebase.

What is it you're suggesting a simple organization should do instead of a "monorepo"?

[tmpz22]

> To me, it just means "keep all the code in one repo, instead of trying to split things up into different repos."

To me, and perhaps more from a Devops-like perspective, mono repo means "one repo many diverse deployment environments and artifacts often across multiple programming languages".

Im advocating against the Google/Stripe situation of a singular massive repo with complex build tools to make it function - like Bazel. I think sometimes small organizations get lured by ego and bad cost/benefit analysis into implementing such an architecture and it can tank entire product orgs in my experience (obviously not for Stripe, Google, etc.).

[jvolkman]

A monorepo doesn't require multiple programming languages or Bazel. But once multiple programming languages are involved, the complexity exists regardless of the chosen tooling. With multiple repos, that complexity is just pushed elsewhere like the CI system.

[ahtihn]

How is a mono repo the simple solution compared to one repo per independently releasable component ?

All the tooling is much easier to use when each application has its own repo.

[jrochkind1]

The argument would be that for simple organization, dividing things into independently releasable components is less simple than just having one app. I think that's what most simple organizations do, no? Why do you need the complexity of independently releasable components for your simple organization? Now you have to track compatibility between things, ensure what version of what independently releasable thing works with what version of what independently releasable other thing, isn't that added complexity? Why not just have one application, isn't that simpler? You don't need to worry about incompatibilities between your separately releasable things -- every commit that passes CI on your single repo means all the parts are compatible (sans untested bugs).

Usually it stops being "simpler" at a level of organizational complexity or code size where it becomes a mess. The "monorepo" is the attempt to do what everyone was just doing anyway for simple orgs with simple codebases, but keep doing it at huge sizes.

[oftenwrong]

The monorepo vs many-repos discussion often hits upon so many implied factors, but it's only really about how source code is stored.

It doesn't necessarily indicate much about the deployment model. You can have many separately releaseable things in one repo, and you can have one independently releaseable thing based on the sources of many repos.

Monorepos enable, but don't require, source-level co-evolution. Or maybe a better way to put it would be: many projects can have a shared history. Many-repos require independent source-level evolution. In the open source world there is no real choice: every project wants to be independent. The authors of a given project can do what they wish with it.

One weird thing to think about is that monorepos can accomodate many-repo style workflows. You can still develop projects completely independently within a single repo. Of course you can store separate projects on separate revisions, which would be weird. An even weirder approach would be having all projects in a given revision, but have totally independent builds, no single-version policy, no requirement for atomic compatibility, et cetera. These are all things that are often imposed for monorepos, but that are also not requirements. Basically, you can treat each project as independent even if their sources are stored together. I don't think there are any reasons to actually do this, of course.

[zten]

If you're living in the same dysfunctional world I am, then maybe your organization split things into repos that are separately releasable, but are conceptually so strongly coupled that you now need to create changes on 3 repos to make a change.

[Lio]

> independently releasable component

I think this is the key phrase in what you've written. Quite often I've seen teams insist on separating things that cannot be released independently due to some form of coupling.

You end up with people talking about a particular "release" but not really knowing 100% what's in it and then discovering later that something is missing or included by mistake.

IMHO it's much easier to keep it all in a single repo and use the SHA value as a single source of truth when discussing what's in it. I don't really work on huge codebases though so your mileage may vary.

[oceanplexian]

> You end up with people talking about a particular "release" but not really knowing 100% what's in it and then discovering later that something is missing or included by mistake.

If your devs couldn’t be bothered to pin versions that was never a tooling problem. You don’t need a 500GB Git repository with every vendored component to know what’s in your code.

[Lio]

Equally, if your team is going to store 500GB vendored components it doesn't matter if that's all in one place or smeared many repos. You still have the same issue.

[epinephrinios]

Absolutely, I worked on tech behemoths and smaller companies. The dev experience was significantly better when all development was local. I even worked on initiatives to move development away from the cloud, and although other devs were skeptical, they ended up loving it.

[Eridrus]

I think we don't have good solutions for scaling down prod.

Our relatively simple prod architecture has 5 containers & a hosted database (so 6 containers when run locally), and any less would impact our product goals.

I still find running prod locally valuable, and is the most common way anyone does development here, but containers are fairly heavyweight when you want to run everything on one machine. It's also impossible if you have parts that need special accelerators to get good latency, etc.

If you're willing to build everything from scratch, you can have a framework that seamlessly lets you build conceptual services and then separate the physical deployment concerns, like Google has and sometimes even uses. But for the rest of us where we're clobbering together a bunch of different technologies, that's a luxury we can't really afford.