Hacker News new | ask | show | jobs
by ryanfitz 4231 days ago
I've only briefly read over the documentation, but this service seems to not follow deployment best practices that aws and others such as netflix have been talking about for years. Specifically the pattern of pre-baking an ami with your current version of the app you are deploying and any other needed software completely installed on the ami and then having an autoscale group be able to boot that ami up in a few seconds and start working. This greatly helps with scaling up, doing rolling upgrades and also very easy rollbacks.

The CodeDeploy service seems to operate by you manually launching base ec2 instance with a code deploy agent and then this agent will checkout your git code on the live instance, run any provisioning steps and then if things break somehow rollback all that work, still on the live instance.

I'm sure this is still a big improvement to companies who are manually sshing into servers and running deployments by hand, but as someone who pre-bakes ami's and does rolling upgrades with autoscaling groups this service seems like a step backwards.

5 comments

I've been working on the CodeDeploy Integration here at Codeship and have been working with the service for a bit (as a preface on my thoughts)

While Immutable Infrastructure is also in our opinion (and I've written about this extensively) the way to go in the future updating systems in place is still the primary way to deploy systems and will be for a while. By providing a centralized systems to upload new released and manage the deployment (how many instances get the new deployment in which timeframe) you can take away some of the security problems of opening up ports for access and potential deployment errors where the SSH connection dies.

Especially when deploying into a large infrastructure connecting into each instance for update becomes painful. That's where an agent based services like CodeDeploy is really powerful and removes the single point of failure that is the machine/network that you deploy from.

With ElasticBeanstalk, Opsworks and Cloudformation they now really start to surround all the deployment workflows.

Definitely a great service that will in my opinion become very important to many many teams. You can also read more about our specific integration in our blog: http://blog.codeship.com/aws-codedeploy-codeship/

In place update is useful in the success case - agreed.

In the failure case however, even with a fleet of only 20 instances, a rolling update that has issues after the 10th instance puts you in a world of pain.

Have you written anywhere how you guys deal with operational monitoring (eg. Boundary, New Relic, etc.) when you're spinning up brand new instances all of the time?
A bit: http://blog.codeship.com/lxc-memory-limit/

We use librato for monitoring our build server infrastructure and mostly only look at max/min values for metrics that could mean trouble. Generally we're able to separate data of different instances by their instance id so we could look into them individually.

We use NewRelic for our Rails application on Heroku and pump Heroku data into Librato as well (we love data and metrics)

And of course you can always send me an email to flo@codeship.com with questions.

I have used Stackdriver (http://www.stackdriver.com/) before and it works good. It can get a bit pricey. They got bought by Google few months ago -- something to watch out for. I really had a good exp with their product.
We use stackdriver very heavily - but also take a look at SignalFuse.
I am wondering why you guys don't see CodeDeploy as a competitor to Codeship? or eventually becoming something like Codeship?
Yep, this is Amazon reaching out to those who won't/can't take the 'immutable infrastructure' approach.

It's fair to say if you're already doing things in the way you describe, this service isn't for you.

Baking everything into AMIs is the "right" way to go, but if AWS had a supported, hosted git server inside their network to push into, I'd rather have the speed of deploying from git, and only bake AMIs when necessary for system upgrades.
...and it looks like one's in the works: https://aws.amazon.com/codecommit/
I have the same opinion after reading over the product page and glancing at the documentation.

At my org, we currently use a very similar method of deployment that AWS CodeDeploy seems to provide, except that we wrote it in house and use some python fabric scripts to do it. It sounds like AWS CodeDeploy would help us to "AWSify" this method of deployment and also have some nice benefits of health checks/rollbacks, UI based management, and a deployment history log. However, we would essentially still be maintaining/writing bash/python scripts to do the heavy lifting.

However, we are in the process of moving to a method of deployment which uses SaltStack to prebake an AMI with everything that is required to run in production and then use Netflix's Asgard to manage the deployment of the prebuilt AMIs. We are very excited about this method of managing deployments to take advantage of AutoScaling groups and using a well defined/tested stack of tools.

Interested to hear thoughts from others using a deployment approach that they think would benefit from AWS CodeDeploy.

> The CodeDeploy service seems to operate by you manually launching base ec2 instance with a code deploy agent and then this agent will checkout your git code on the live instance, run any provisioning steps and then if things break somehow rollback all that work, still on the live instance.

Deploy and rollback doesn't happen on live instances. (Group of) Instances are taken off service for rolling deployment and you can configure what percentage of your fleet should be deployed to at once.

Looks like an improved version of OpsWorks