[csdvrx]

> I would argue that all deployments (no matter how small) should have configuration management.

I would argue that in most cases, they don't need anything but some documentation explaining what was installed, and why.

I will take a word file with screenshots over a broken script in an obscure language every single time.

> but there's no reason not to use something like Ansible that's designed for infrastructure automation,

There's a big one: my time isn't free.

If someone is willing to waste money on that, sure, I'll be happy to bill them for their extravagant tastes (but only after having done my best to explain them it's a waste of money)

And still, I will think about the next person that may have to maintain or tweak whay I wrote, so I will also leave a document full of screenshots in case they don't know ansible or whatever new fashionable tool that the client may have specifically requested.

> it's faster and the resulting system is of higher quality.

Not everything needs to be of high quality.

Forgive me if I'm assuming your gender, but I see a lot of black-and-white thinking among male sysadmins/devops: it's good or it's bad, it's high quality or it's not.

I prefer to have a "sufficient" degree of quality: if a checklist is enough, I will not waste time writing a script. If a shell script is enough, I will not waste time writing proper code - and so on.

> Over time you will also accumulate a library of bits and pieces that you can copy over to new setups, further improving your speed and quality

Except you assume a continuous progress, without any change of scope or tools, and with the tools themselves never evolving. It doesn't work like that: over time, you will accumulate a bunch of useless code for old versions.

Even small inconsequential changes (like unbound in debian 11 requesting spaces before some options, which wasn't the case before) will take some time and effort. Why waste your energy one one shots?

The do-nothing approach argues that you should avoid premature optimization, which strikes me a good approach in software in general.

[brulard]

Such a needless dismissive use of "male" word here. Devops guys I work with are very reasonable and I have not seen a female one in my whole career.

[loxias]

> Forgive me if I'm assuming your gender, but I see a lot of black-and-white thinking among male sysadmins/devops: it's good or it's bad, it's high quality or it's not.

Male lifetime linux nerd here, who started as a sysadmin, checking in just to say that I agree with everything "policy related" in your comments on this article. Knowing where to tune the knob between "high quality"/"good architecture" vs "can i just get this done now and move on?" is difficult, at least I don't know how it could be taught other than experientially.

IME, the predilection to see things as black-or-white is more correlated with age, than gender.

Anyway, "not all men". :P

[csdvrx]

> checking in just to say that I agree with everything "policy related" in your comments on this article

Your nick seemed familiar - now I remember, I read your great comment in "I just want to serve 5TB" earlier today!

I also agree with everything you wrote about simplicity in software development: I'll take almost every time some dirty php scripts running baremetal over Docker + Golang + Kubernetes + Terraform + Gitlab + Saltstack + Prometheus + the new fashionable tool because with so many parts now begging for attention, nothing will get done quickly - if we're lucky and something gets done.

Knowing where to tune the knob is indeed very difficult, and I'm afraid most people now are just doing a cargo cult of whatever google does, except they are not google, and they don't understand the tradeoffs or the possible alternatives.

But at the scale of most companies, it's a folly to sacrifice flexibility and simplicity to some unachievable desire for software perfection!

It's also a very costly hubris: I have been asked way too often to improve the performance after having thown very expansive hardware at the problem, that still performs miserably due to missing the big picture.

The solution was almost always removing the useless parts, or when trying to disentangle the architecture astronaut fancy mess would have been too costly, start from scratch with a saner design: most recently, I replaced a few hundreds java files (and test and stuff) by about 10 lines of bash, and 20 lines of awk.

My work is not fancy, but it works, unlike the previous solution that was going to be ready the next month, every month, for almost a year...

To all those who want to do things like google, maybe apply there instead of over engineering/polishing your CV with fancy keywords at your employer or client expense?

> IME, the predilection to see things as black-or-white is more correlated with age, than gender.

I had noticed this weird pattern, and it was my best explanation even if I didn't like it much, because it's sexist.

But your version seems more plausible (Occam's Razor!), so thanks a lot for taking the time to post!

[chousuke]

I do not think automation is "premature optimization", nor do I that think everything needs to be high-quality; I did not say that. I do think, however, that everything you do should be of acceptable quality.

And for me, having configuration management is the minimum level of acceptable quality. It's simply not possible to have acceptable quality of a system without some form of configuration management. I can't recall a single instance where I (or anyone else involved) ever said "wow, this unmanaged mess sure works well" :P

In some cases, the management can be as simple as a comment in some script explaining some part of the process was done manually, or simply a periodic snapshot backup of the server that can be restored when the configuration is broken, but the point is that a process must exist and it must be consciously chosen.

Free-form documentation is not an alternative to configuration management either; if you can document your configuration in a wiki, you might as well put it in a git repository in the form of a script or a template.

When done properly, It's the exact same amount of effort, except when you use automation tools, the documentation is in a format that's not ad-hoc and can actually help do the things it documents instead of requiring a human to interpret them (possibly introducing mistakes). "Setting up" Ansible requires literally nothing but a text file containing your server's hostname, and SSH access, which you already will have.

Also, I don't know where you got the idea that I would somehow assume unchanging scope? I am the first person to throw away useless code and tools; I consider code my worst enemy and it's practically my favourite activity to delete as much of it as I can. If some piece of automation is no longer fit for purpose, it gets rewritten if necessary. Throwing away code is no big deal, because the tools I use allow me to get things done efficiently enough that I can spend time refactoring and making sure the computer properly handles the tedious parts of whatever I'm working on.

Your unbound example is something that is trivially solved with configuration management. After an upgrade, you notice your configuration does not work, navigate into your git repository, update the template, and then deploy your changes to any server that happened to be running unbound using that same template (because you might have redundancy, if you're running DNS). If you make a mistake, you revert and try again. There is no manual ad-hoc process that comes even close to allowing you to manage change like this, but it is trivially enabled by existing, well-understood automation tools.

[csdvrx]

Your definition of "acceptable quality" is my definition of "overengineering".

It does not take the same amount of effort, if only cause you mention how for unbound, you have to update the template.

For one shots, this is overkill.

[randomswede]

For "truly one-shot", you're right. But a "truly one-shot" is not a production machine, it is a test bed, informing what the eventual production machine should look like.

Because even if you will only ever have a single production machine, it will have something go horribly wrong with it and need recreating from fresh hardware (or from a fresh VM or whatever).

I guess, if you're cloud-based, you could turn your finely tuned test box into a template, then you have something that is (effectively) scripted.

[chousuke]

Leaving aside all the other benefits and even if you never need to rebuild your system, having some sort of IaaC automation in place allows for extremely powerful change management. When your system is defined as code[0], change over time can be reviewed with a "git log -p", which definitely beats searching through ticket comments or ad-hoc documentation and attempting to reconstruct the history of change.

It's a no-brainer nowadays that software should be developed with version control. I don't see why infrastructure should be treated differently.

[0] Ansible playbooks are code, no matter what some people may think. It's a declarative'ish programming language with a silly syntax.

[chousuke]

There's no such thing as an oneshot if you're creating a system that someone will actually use and depend on.

All systems have a lifecycle, and even on a "trivial" system you have backups, access, monitoring, logging and security maintenance to worry about even before you consider how installing any useful software affects those things.

There are exceptions to any rule, of course, and I did in fact create a system where the configuration management is a snapshot backup just two weeks ago; but that system has no data on it, its lifecycle is expected to last for less than a year, and if/when it breaks, a backup restore can be performed without any additional considerations. It was also an emergency installation into a network that's not easily accessible with SSH, which is why I did not just use Ansible from the start.

I thought it would be a oneshot, but I did end up having to create a second instance of the system a few days later, fortunately with less emergency :P

Still even ignoring that, I fail to see what could possibly be overkill about literally 3 small files in a git repository. You call "overengineering" what is to me "5 minutes of effort with extremely relevant upsides". That's literally how much time it would take me to create a playbook for unbound if I already know what the configuration needs to look like; probably less, but most of the time will be lost to context-switching overhead.

My point being, most of the time will be spent actually configuring the software and the automation overhead is nothing in comparison compared to the value you get from it, and that's why I generally automate things by default: It provides more value than I put in effort.

When you start of learning configuration management and infra automation tools, there's a learning curve; in the beginning, you will be "wasting" time learning (what a silly statement) how to use your tools effectively, but with practice, you will learn how to effectively use the tools and where to apply them and how to approach managing specific kinds of systems such that over time, using the automation tools is simply easier and faster than doing it manually. That's what I meant when I mentioned "higher quality" earlier; you get it for free, with no effort, once you've put in a bit of practice first. It just sounds to me like you're arguing against doing things well in favour of doing things with strictly inferior tools.

[rgj]

> Not everything needs to be of high quality.

But stuff connected to the internet needs to be or it will be compromised before you even finished installing it.