[giaour]

Amazon has Brazil versionsets and workspaces, which solve many of the same problems monorepos do. I really liked the way those extra resources let you organize "ad-hoc monorepos" from smaller repositories, but it's infrastructure that I haven't seen elsewhere.

Since leaving Amazon, I've mostly worked with monorepos and wouldn't go back to multirepos without Brazil-style tooling.

[ghoward]

Hey, I'm hoping you can answer some questions for me.

I'm building a build system and a VCS (separately). I want to do it right.

Could you explain to me what Brazil is? Is it the build system? [1] Or is it the VCS that Amazon uses?

If it is the build system, then it appears that versionsets are literally just a list of dependencies with their versions to use for a build. Is that correct? If not, or if you can give me more detail, what are versionsets, exactly?

Also, what are workspaces? Does this quote from one of the comments on the link match?

> A workspace consisted of a version set to track and any packages that were checked out.

[1]: https://gist.github.com/terabyte/15a2d3d407285b8b5a0a7964dd6...

[dastbe]

there's nothing particularly magical and a lot the behaviors are actually similar to bazel, in ways. a version set is a directed acyclic dag of packages linked by dependencies. These edges comprise (more or less)

* normal deps

* compile/test deps (i.e. non-transitive)

* runtime deps

* tool deps (non-transitive, but also non impacting on the closure resolution algorithm)

version sets allow for multiple versions of the same package to exist in them (ex foobar-1.0 and foobar-1.1), which has some benefit but in practice is just painful.

dependencies are defined in a capital-c Config file. when you run brazil, it does a few things

* it resolves all of the tools and makes them available on your path

* it sets up some environment variables

* it invokes your a build command defined in the Config file

Config files can also declare outputs (there's also some canonical outputs), so you can use a query tool to ex.

* get all jars in the runtime closure

* generate a symlink farm of all client sdk configuration files

the results of your build go into a build directory, and when you want to generate the runtime for a particular package, it will symlink together the outputs of all packages in the deps + runtime closure, which you can do on demand.

version sets are updated by building new packages versions into them, which will rebuild the version set to make sure all builds pass. if they do, a new "commit" will be put onto the version set. you can also merge from one version set into another, where you can get packages and their associated dependencies merged in along with a full rebuild.

[ghoward]

Oh, wow, thank you for the detail!

[giaour]

I am not an expert on Brazil, but the link you shared matches my recollection.

It's important to remember that Brazil covers a lot of ground, and many internal tools at Amazon are external to Brazil but rely on it and the way it organizes resources. So I (or others on the internet) may incorrectly call something Brazil if it's part of the larger Brazil ecosystem.

Let's start with a repository and build outwards:

You have some code (let's say it's a Java library) that does something cool. To compile it, you add a Brazil file to the repository root. This file specifies what Brazil packages your code depends on, how it should be built, and what kind of artifact it will produce. Once that file is there, you can run "brazil-build" to produce a Brazil package (which is just a jar with some metadata).

You want to use this library in a web service, so you check out both repositories in a single workspace. Every workspace has a source versionset where it fetches dependencies, but if the code repository for a package is checked out locally, "brazil-build" will build and use the local version instead. You make some changes to the library and web service, then test how they work together by running the web service from within the workspace folder. This ensures that it is using your local modifications to the library repository before those changes have been merged.

Once you're satisfied with the change, you open a PR with a brazil-integrated tool that can show changes to multiple repositories as an atomic change (a "change set"). The CI system for this tool uses a Brazil workspace to make sure your update code packages build together and that all tests pass.

If the PR is approved and you merge to main, there is probably a pipeline watching your repository for changes that proactively rebuilds one or more version sets based on the merged change set. Any package in the version set that depends on your library will be rebuilt using the changes from your change set. So while a versionset is implemented as a list of packages at specific versions, it's best to think of a versionset as more or less equivalent to a monorepo containing all packages on that list, since changes that cross package lines can be built into a versionset in an atomic unit. (This is very helpful if you need to push out a breaking change.) A package can exist in multiple versionsets, which is of course impossible with monorepos.

[ghoward]

Thank you for describing the workflow. This makes it so much easier to understand them!

[dastbe]

very much this. there's also a cultural understanding that libraries/fat clients are equal or worse in terms of maintainability compared to services. equal in the sense that you have to treat them like a service, worse because it's easier to mess up and you don't get to control your rollout strategy.

EDIT: Though while most teams have gone towards many smaller packages for their applications, I suspect that most would be better served by a team level monorepo. That gets you all of the benefits of monorepo locally and all of the benefits of manyrepo globally, and unless your project hits the ~200+ developer mark maintaining things will be stay tractable.