Hacker News new | ask | show | jobs
by wldlyinaccurate 4338 days ago
Everybody seems to be taking about moving away from Puppet lately. Maybe I just don't do anything sufficiently complex with it, but I've never had any problems or gripes with Puppet. 99% of the time it seems like the thing I want to do has already been done in a well-written module on the forge.

The author seems to cite two main reasons for wanting to move away from Puppet: their codebase was large and badly structured, and their techops team didn't know Puppet well enough to manage it. Neither of these sound like problems with Puppet itself -- they're certainly not unique to Puppet. I'm not convinced that moving to a newer, less mature technology (which I assume techops don't know well either) will solve these problems.

3 comments

There's definitely more reasons. I didn't want to detract too much from the topic of the blog post when I wrote it, since the post is already obscenely long.

Puppet doesn't have native support for a lot of things, which require us to either implement it in puppet's DSL, or in custom ruby, which the upstream won't take. For instance: git, gems, pip, virtualenv, npm, etc. etc..

Puppet doesn't have looping. I'm always told: "Iteration is evil. Puppet is a declarative language and if you're needing to loop you're doing something wrong." But it's simply not true. Looping making things insanely simpler.

Puppet isn't executed in order, even for the same service in the same environment across systems. You have to very diligently manage every require for ordering, and no one does it right. This had lead to systems unable to run first runs really often, which causes problems with autoscaling. I don't enjoy spending my time cleaning this up often.

Puppet's DSL is full of little gotchas that constantly cause issues for developers who aren't very familiar with Puppet.

Half of our team was very familiar with Puppet. If you look at my blog, quite a few of the older posts are about Puppet. I worked on the puppet infrastructure at Wikimedia Foundation for a long time, and released all of the puppet code as open source (they have 60k+ lines of puppet).

I'm a little sad because most of these issues (as I understand your description them) are already fixed or well underway :( It's probably too late for your specific case but I'd like to reply anyway since a lot of this is "conventional wisdom" based on old information. Full disclosure: I'm the product owner for Puppet and before I worked here, I ran it in large-scale production since 2008.

Not quite sure what you mean by 'native support', but gem and pip package providers are built-in. there are high-quality modules for git (puppetlabs-vcsrepo), virtualenv (stankevich-python), npm (puppetlabs-nodejs), etc -- it's a design decision to move much of this into modules and out of core so they can iterate faster.

Totally agree that loops make many language constructs much easier, and there's looping in the rewritten parser -- https://docs.puppetlabs.com/puppet/3.6/reference/experiments... -- which will be on by default in the next semver major.

While the model definitely wants you to describe relationships between resources if you need to send subscribe/refresh messages, there's toggle-able ordering algorithms that will let you run them in manifest order -- I blogged about it here: http://puppetlabs.com/blog/introducing-manifest-ordered-reso...

The parser and evaluator are undergoing a total rewrite to be an expression based grammar, which is explicitly to make better definition around the language and eliminate the gotchas -- https://docs.puppetlabs.com/puppet/3.6/reference/experiments... (this will also be the default on the next semver major)

--eric0 / @ahpook

Native support for things is irrelevant cause you can use modules from forge, and the community is the largest of all other CM tools around, so I hardly believe that you lack something there.

Actually, you can circumvent lack of looping with defined types and calling them with array. In my opinion if you need loops in your infrastructure code you're doing something wrong.

Saddest thing is that from all the people who brag about migrating away from puppet online nobody actually mentioned some of the drawbacks that are REAL and present - and not even discussed in Puppet community - like lack of simple search function vs complexity of exported resources... that means that people are moving away for reasons different then functionality alone...

Another real issue is the slowness of compile process, which happens on the master. But it's OK for "smaller" deployments - like if you don't go above 10-20k nodes.

Had the same thought. Puppet code was bad and no one knew Puppet. Seems like a fine reason to move. But could be the other way around. Could be Ansible code is bad, no one knows it, lets move to Puppet!
The cool kids have a new fad so you're not cool unless you dump puppet. No technical reason at all as near as I can see. Its pretty much the same as "Perl hate", why do we hate Perl? No reason at all, other then being cool means hating Perl! Very middle school social dynamic.

My puppet manifests is 16K. My modules is larger but I've got some large files stuck in there (long story)

There are meta questions like:

What are you doing with 15000 lines of puppet? I have a couple thousand and feel a bit over extended, like why am I doing this.

How are you replacing ten lines of puppet with 1 line of alternative when all I'm seeing in the examples is replacing 3 lines of

group { "logusers":

    ensure => "present",
}

with

- name: Ensure groups exist

  group: name={{ item.key }} gid={{ item.value.id }}

  with_dict: users
Like, where is the big win where those 3 lines of puppet are being turned into 0.3 lines of Ansible?

There is also the question of why I'd configure individual groups on individual machines instead of just tossing it in the LDAP once, probably by hand. Or distributing a system wide /etc/groups much as I used to share a division wide emergency /etc/hosts (like, this is the minimum /etc/hosts required to conveniently fix DNS if DNS breaks).

(edited to add actual numbers. I have ldap and getent group | wc -l reports 76 groups. I could replace that with 76 groups * 3 lines per group plus a blank line between entries = 304 lines of hand maintained code. But in 3 lines I could distribute a golden /etc/group to all machines. Or in a few more lines I could make all my machines use LDAP and get passwd and some other stuff centrally controlled for free (and yes I use ldap for passwd and no I use kerberos for auth, so passwd just holds home dirs and stuff like that). So I could write hundreds of lines of puppet to get out of editing one golden group file or get out of running ldap, but the alternatives are so much easier...)

There exists a meta question of allocation of resources. You can do "everything sysadmin" in puppet. Or make a universal does it all gold image that is well backed up and enables or disables parts of itself based on role and never automate its configuration at all, just spin up images and give them "special" hostnames and they sort themselves out. Or not automate trivial parts. Or place some weirder config stuff in a shell script technically not part of puppet other than being distributed, run, and tested for error free operation. Or a mix across all. So I could see a "gentoo-like" start with an official distro image and use nothing but puppet to do everything taking 15000 lines of code, maybe. But that sounds hard... do it a different way, no need for different tools.

I have a category on my blog dedicated to LDAP: http://ryandlane.com/blog/category/ldap/ I used it very heavily at Wikimedia and had very nice integration with Puppet. In general I think it's good to avoid LDAP if possible. It adds a point of failure and assuming you're not managing thousands of users (we were handling about 5k users in Wikimedia Labs), it's generally more work than managing users in Salt/Ansible/Puppet.

We didn't save a lot of lines of code replacing the user/group code with Salt. We saved a lot of lines of code by using native support for git/pip/virtualenv/npm/etc, which were implemented as a mix of custom puppet DSL and ruby.

We could have likely saved 3-5k lines of code from a puppet rewrite from scratch, but it still wouldn't have been as simple as the Salt or Ansible code.

PIP support is built into puppet, not sure about other stuff but since you missed this one there's a chance u missed others too.

So basically you don't have any valid technical points against puppet other then Salt seems easier for you to use. Which is ok.

"Its pretty much the same as "Perl hate", why do we hate Perl? No reason at all, other then being cool means hating Perl!"

Ummm, no. Believe me, having worked in a PERL shop for nearly 10 years, I was ecstatic to start working in Java (!). And I have zero regrets. It's nothing to do with being "cool" or any other pointless patronising insults to other developers. It's that badly written PERL is the worst excrement ever to have been smeared on a computer screen. Yes it can be clean, clear and readable, but only in the hands of an experienced expert (using "modern PERL", which only started to exist ~6-8 years ago) backed by stringent code review and consistent team practice. Without the infrastructure in place, you end up with reams of unmaintainable, slow, buggy, eye bleed code. I've seen plenty of PERL from plenty of projects and plenty of different developers in different institutions and companies and the vast, vast majority is crap.

The last bit of code I had to rewrite (5 years effort by a weak PERL coder, replaced in 3 months in Java, maintained by someone with no previous Java experience quite happily) would have made you cry. I have never seen anything like it in any language (note, I don't work with PHP either), and I don't believe such a steaming mess would be possible in any other modern programming language, let alone from someone who had been a professional programmer for 10 years.

To me, your attitude is the problem the PERL community has. The language deserved the bad reputation. Until modern PERL appeared, it was almost impossible for a beginner to produce anything like good code. It is possible now (I have seen beautiful clean PERL), but instead of trying to educate and bring people back to the fold, the community has a massive chip on its shoulder, refuses to admit the problems PERL has (still), or that it ever had any, and proceed to insult everyone else. If PERL hadn't learnt from the trends in other languages (esp a decent OO system, Moose), it would be literally dead by now.