[moe]

Sorry, but I wonder what you're talking about. Package managers are designed exactly for this purpose.

It's not even hard to leverage the power of apt (the most advanced of the bunch) for your own deployments.

In essence you setup a local mirror and add that to the sources.list of all your hosts. Then you learn how to roll your own debs and push them to the mirror. The actual deployment happens via apt-get - which can be triggered by cron, a shell-script, puppet, a sweaty admin at 4am, or whatever fits your bill.

Working with the system this way instead of along with it (or even against it) has various advantages. Most importantly you get proper dependency management. Need to roll back to app version 2 from version 3, whereas version 2 depends on an older version of foo? No-brainer, apt takes care of that for you, both ways, also in much more complicated cases. Need a package or package version that's not in the official repos? No problem, roll your own and make your application package depend on that.

With a bit of elbow grease you can also have it mangle your database and other auxiliary infrastructure appropiately, within the respective pre-/post-install scripts.

Fabric and capistrano are just expressions of the old "if you don't understand you're doomed to reinvent, poorly" meme.

[delano]

I agree with you. But now I want to launch and release to several EC2 and Rackspace machines, in parallel. apt doesn't help with that. It also doesn't help with releasing to multiple machines simultaneously (including different types).

If I have 5 debian machines that need to be updated, I should be able to do that with a single command and it should happen in parallel. The same applies if I have 5 debian machines and 5 red hat machines (etc...). I'm advocating a tool that is aware of the existing system specific package managers rather than a replacement of them.

[moe]

I agree with you. But now I want to launch and release to several EC2 and Rackspace machines, in parallel. apt doesn't help with that.

Ofcourse it does. What makes you think it doesn't?

If I have 5 debian machines that need to be updated, I should be able to do that with a single command and it should happen in parallel.

reprepro -Vb . stage1 myapp_2.0-1.dsc

That drops a new pkg onto the mirror where the staging hosts pick it up within one minute, from cron. I could use the "live" distro instead of "stage1" to roll it out to production. We use sections if we want to limit the push to individual groups of hosts.

The same applies if I have 5 debian machines and 5 red hat machines (etc...)

If you mix linux distributions in a production environment then you have bigger problems to resolve first.

I'm advocating a tool that is aware of the existing system specific package managers rather than a replacement of them.

Those who don't understand are doomed to reinvent, poorly...

[delano]

I think we mostly agree, we're just looking at the problem from different directions.

Of course it does.

Can apt launch EC2 instances and execute scripts (that are not part of the package) before and after installation? Can it update security group settings and request and assign static IP addresses? My understanding is that apt does not help with these problems, so we write scripts or use tools like Fabric to do this. These scripts/tools are aware of the package manager in that they call the commands to make things happen. This is the level I'm talking about at which there are open problems.

If you mix linux distributions in a production environment then you have bigger problems to resolve first.

In an ideal world this is true, but it does happen. For example, one vendor my require a specific type or version of OS from the rest. A business may also choose to change the OS from one release to the next.

It's important to be aware of what is possible and account for it ahead of time. Again, I'm not advocating not to use apt or yum or rpm. I'm suggesting that it's helpful to not tie your process to a specific one unless you have complete control over the environment, now and for the foreseeable future.

[moe]

Can apt launch EC2 instances and execute scripts (that are not part of the package) before and after installation? Can it update security group settings and request and assign static IP addresses? My understanding is that apt does not help with these problems, so we write scripts or use tools like Fabric to do this.

Well apt does not launch EC2 instances, you launch them, after you defined their role in your central configuration server.

The first thing a launched instance does (in rc.local) is "apt-get install bootstrap". The bootstrap package contains everything a node needs to come alive. Ours consists of not much more than a script that immediately runs via the post-install hook. This script is where the magic happens, it connects to the "hivemind" and gathers the configuration data, based on the node name that the instance was parametrized with at startup. According to the role it is asked to assume it will install the appropiate application packages (we call them "logic bombs"). For sanity it makes sense to just name the packages after the role. We have packages for "faceplate", "db", "queue" and such.

The packages will depend on other packages as needed and most of them contain pre-install hooks for initialization tasks (e.g. mount an EBS volume for a database node, claim an elastic IP, mangle DNS, etc.).

Well, long story short, I think the key mistake of capistrano and fabric is to assume Push where you really want Pull. Once that is realized life becomes much easier.

My understanding is that apt does not help with these problems, so we write scripts or use tools like Fabric to do this.

Apt is ofcourse just one part of the toolchain and scripts will always be involved either way. My point is that a toolchain built around apt most likely has no need for something like fabric. Fabric is just not a very useful abstraction in a scenario involving more than a handful of hosts.

In an ideal world this is true, but it does happen. For example, one vendor my require a specific type or version of OS from the rest. A business may also choose to change the OS from one release to the next.

Well, these are problems technology can't fix. These are problems only the HR department can fix.

I'm suggesting that it's helpful to not tie your process to a specific one unless you have complete control over the environment, now and for the foreseeable future.

There is a word for systems where nobody assumes "complete control": abandoned.