Hacker News new | ask | show | jobs
by mark_l_watson 4392 days ago
For at least the last 15 years I have almost always done 'devops' myself (before it was called that). Managing deployment goes with designing and building systems.

My problem is that I am sort of stuck in the past. Whether I am using VPSs, AWS, or rented physical servers, I have only a partially automated way to set up servers. This scales to small numbers of servers just fine, and that is mostly the world I live in, but I need to improve my workflow. This really hit home yesterday when I had to upgrade a Haskell GHC/platform because I tweaked a Haskell app making incompatible with an old GHC 7.4.* setup on an older server, and ended up wasting some time before fixing things.

Working as a contractor at Google last year was an eye opener. I really loved their infrastructure. Eye opening experience.

Docker seems like my best path forward.

3 comments

You have hit the nail on the head. The thing people want in the real world is improved workflow.

In the real world, everyone wants infrastructure to have the same sexy qualities: automated deployment (CD/CI), automated scaling, automated failover/high availability, automated service discovery (read: functional service topology dependency resolution), security, resource and capacity planning support, real time least-cost-overhead provider selection for third party infrastructure providers meeting some disparate set of performance requirements, etc. Unfortunately, it's not an easy problem area to deliver a one size fits all solution to.

Docker doesn't really have most of that stuff in scope yet, even vaguely. Actually, it seems to have a really weird scope: it wants to wrap different LXC implementations and other container-style unix environments (potentially supporting non-Linux platforms) but doesn't want to deal with managing the host systems themselves, having - kind of, for practical reasons (though not entirely!) - outsourced this to CoreOS (ie. some particularly specific configuration of a Linux host system).

Whether all of this recent Redhat/Google docker bandwagon jumping will amount to any real solution remains to be seen .. Google AFAIK effectively runs its services on fat clusters made of commodity hardware, organized in to segments ('cells'), running highly customised Linux distributions, and so does Redhat where HA is required. I'm pretty familiar with these configurations as I do this myself. So will we ever see meaningful support for other OSs? Other distros? Physical systems via PXE to support these clusters? Hypervisor guests managed with the same developer and operations workflow?

My wager is not soon, at least in a manner that everyone agrees on... Google will keep doing its thing (using its unlimited supply of internal, world-class nerds to deliver and manage services on their custom OS in a custom way because saving 1/2c a month per machine pays ten world class nerd salaries at their scale), Redhat will keep doing its thing (selling prebuilt systems at expensive prices that still comfortably undercut the likes of IBM, pretending they are manageable, but actually rejigging the whole infrastructure every system release leaving little in the way of realistic upgrade paths without expensive consulting) and you and I will be left wondering where that magical docker solution went that everyone was talking about in early 2014.

Google values devops / solutions, so I think we'll see some interesting announcements over the next 12 months.

Here are a couple of notes..

Deployment manager - https://developers.google.com/deployment-manager/

Saltstack integration - https://www.youtube.com/watch?v=0dOXbhenFl0

I agree, but that's like saying "I think the world will continue turning". Obviously, yes!

The deployment manager sounded interesting but I'm not seeing any support for arbitrary platforms (in the OS sense), or infrastructure providers (in the 'run it on my own hardware, or someone else's' sense), nor the opsier side (like business concerns separate to technology) of the ops part.

Some thoughts roughly summarised at http://stani.sh/walter/pfcts/

Thanks for sharing your thoughts. Great points and insights.

Following up a bit.. Google just announced Kubernetes[1], an open source container manager. Also, Eric Brewer is now on Docker's Governance Committee[2] to help push for open container standards.

Seems like a good step forward.

[1] https://github.com/GoogleCloudPlatform/kubernetes [2] http://googlecloudplatform.blogspot.com/2014/06/an-update-on...

> My problem is that I am sort of stuck in the past. Whether I am using VPSs, AWS, or rented physical servers

Makes me feel old... I used to "deploy" with cPanel or DirectAdmin.

I agree.. I had a very similar 'eye opening' experience as you did with Google's infrastructure. I am now working on a internal projects as a contractor / partner. Solid.