[omry]

Click let's you build really nice command line libraries, but those are rigid and their structure is hard coded. A good example of what you wouldn't do with Click is logging configuration. You have a bunch of appenders, formatters and loggers and this is just not good with something like Click.

Hydra actually comes with built in Python logging configuration, it's just a part of the configuration object and Hydra is setting it up for the app: https://cli.dev/docs/tutorial/logging

You can override this from the command line (unlikely but could be useful), or completely replace it with your own logging config: https://cli.dev/docs/configure_hydra/logging

This kind of flexibility is a core part of how Hydra build the config, it's not limited to it's internal config (like logging) but extends naturally to the user config as well.

[choppaface]

It might be nice if mlflow adopted something like this as a feature.

[omry]

Tell them to take a look at Hydra.

Hydra has some overlap with what MLFlow is already doing, but I am sure MLFlow can benefit from configuration composition.

Another way to look at it is: What kind of Hydra plugins can be backed by MLFlow. for example, remote launching to any cloud can be implemented as a Hydra Launcher plugin. Currently the included launcher plugin is trivial (local execution), and the more interesting launchers to launch remotely are not open sourced as hey rely on internal FB infrastructure. Supporting the major cloud providers in Hydra through plugins is definitely something I would like to see.