[jzelinskie]

Ah, a shared solution to this problem is sorely needed in the Python ecosystem. When we developed the Quay[0] container registry, we had a bunch of gunicorn processes all with their own Prometheus client registries (the in-memory data structures keeping track of the metrics), but you can't just scrape this directly. We ended up having to adopt a push model and then even then it's tricky to make sure metrics are designed to aggregate properly.

[0]: https://github.com/quay/quay

[llandy3d]

In theory the push gateway is to be used for batch/one of jobs, in limited cases because it had drawbacks

But yeah that's what I'm trying to solve in a different way, registries in different processes are separated but with the "backend" architecture, if they are the same metric they will end up in the same place. Redis can be considered the shared memory for all the processes that together make a single service, when you scrape one of the processes, you get back the correct value. This is possible because redis is single-threaded & the float increment operations are atomic. I'm sure not everyone might want to use redis, so the library implements a "Backend Protocol" that you just have to respect the interface and so you can build a different solution with different toolings, it's pluggable! (This made it extremely easy for me to implement a different backend in rust that might also remove the dependency on the redis client in the python side :) )