[jvolkman]

This is possible with existing general monorepo tools. We use Bazel for Python and other languages. With Python, we have a single requirements.txt at the root of the monorepo, but each target (a Python package, a module, an app, or some combination thereof) depends only on the packages it needs and the transitive packages of its dependencies.

Then if you need to do something like build a distributable archive for your app (say, to deploy it to lambda), you can use Bazel's dependency graph to get only the packages that the app transitively depends on.

[0x008]

What about the opposite case? I have several python packages with one requirements.txt each. But for development I want to have a single “root” virtual env?

[jvolkman]

In that case it seems like you could just have one all_requirements.txt that includes the individual files like:

  -r package_a/requirements.txt
  -r package_b/requirements.txt

etc.

[0x008]

yes I get it. Good point.

Is bazel also a good choice for managing the virtual environments of my developers? Do you have any examples?

[jvolkman]

At this point I would say no, but work is in progress to improve in this respect [1].

At my place of work we use some rules [2] to build virtual envs from Bazel dependencies. It's been a great stop-gap for us and allows us to use traditional Python tooling (e.g. PyCharm) with Bazel-managed dependences.

[1] https://github.com/aspect-build/rules_py [2] https://github.com/cedarai/rules_pyvenv