[achempion]

I find your example simple and easy to understand and I don't see any issues with defining everything explicitly.

I'm using docker swarm/compose extensively for multiple years and configuration can be a bit verbose sometimes but it's very simple system to maintain and write config for.

I think, splitting relatively small configs into smaller bits and using additional abstraction can actually increase complexity of the system compared to having everything in a single file as it's basically write and forget and not something you modify heavily once system is mature.

[Randomdevops]

Well they are examples :)

But imagine the complete deployment looks like 2 monoliths, a dozen supporting webservices, 3 databases, redis, an elasticsearch cluster, ActiveMQ and a mailserver.

Then for a development track you only want to run monolith A (cost/resources/startup time) and the mailserver being MailHog so you don't accidentally send something to real addresses.

But you can't just split the 2 monoliths in separate compose files, as they share services/dbs. If you did that you would have to wire them in manually.

For local development you might only want to run webservice B (which has a db and redis as dependencies)

And indeed if you try to manage several files that need to be combined in a specific way things gets messy...

So then things seems to boil down to starting every service separately and relying on service-discovery, but then the lifecycle isn't tied together anymore. So if I want to tear them all down, how would I know which ones I spun up in this imaginary context?

Say I want to spin up an on demand integration environment for an application, what else do I start too?

[achempion]

> But imagine the complete deployment looks like 2 monoliths, a dozen supporting webservices, 3 databases, redis, an elasticsearch cluster, ActiveMQ and a mailserver.

Make sense, in such case you probably want to transition your production environment from compose to one node swarm cluster (docker running in swarm mode).

Then you just define ingress server (caddy/nginx/haproxy/traefik), probably some service you want to share like a database and use docker stacks (docker stack deploy) where you can keep config for each stack separated.

This approach is very modular yet still relatively simple to maintain once you understand concepts around docker swarm and how docker stacks work.

If you want to be more robust, you can describe your deployment in simple ansible recipes, but I found this approach a bit cumbersome and try to stick with simple readme file for deployment as long as possible.

For more complex deployments I'm toying with an idea of diving into hashicorp nomad but this type of tools requires a dedicated devops team which makes sense for large multi-node deployments when swarm allows you to keep stack very simple as most devs already know docker-compose and the learning curve to maintain swarm clusters is still approachable compared to properly diving into devops world.