[psv1]

Can anyone offer a good guide to DevOps for people who don't directly use these tools but work with engineers who do and would like to learn more? The whole ecosystem of servers, cloud infrastructure (and all of the different offerings there), Docker, Kubernetes, CICD tools etc is a bit overwhelming to get into.

[dsr_]

Sure. None of the things you mentioned are DevOps.

DevOps is two things:

1. Applying the methods of modern software development (version control, automation, DSLs...) to operations (provisioning, config, deployment, monitoring, backups...).

2. Reducing silo barriers between devs and ops groups so that everyone is working together as a team, rather than blaming each other for poor communication and the resulting messes.

Then there are all the DevOps hijacking attempts, such as equating it to Agile or Scrum or XP, or insisting that it's a way to stop paying for expensive operations experts by making devs do it, or a way to stop paying for expensive devs by making ops do it, or a way to stop paying for expensive hardware by paying Amazon/Google/$CLOUD to do it.

No matter what your software-as-a-service company actually does, it will need to execute certain things:

- have computers to run software

- have computers to develop software

- have computers to run infrastructure support

You can outsource various aspects of these things to different degrees. Anywhere you need computers, you have a choice of buying computers (and figuring out where to put them and how to run them and maintain them), or leasing computers (just a financing distinction), or renting existing computers (dedicated machines at a datacenter) or renting time on someone else's infrastructure. If you rent time, you can do so via virtual machines (which pretend to be whole servers) or containers (which pretend to be application deployments) or "serverless", which is actually a small auto-scaled container.

Docker is a management scheme for containers. VMWare provides management schemes for virtual machines. Kubernetes is an extensive management scheme for virtual machines or containers.

A continuous integration tool is, essentially, a program that notes that you have committed changes to your version control system and tries to build the resulting program. A continuous deployment system takes the CI's program and tries to put it into production (or, if you're sensible, into a QA deployment first).

[movedx]

At last, someone who gets it. Absolutely nailed it. Great answer. I never log into my HN account anymore, but for this response I just had to say: yes. Well said.

When you boil the Cloud, DevOps, CloudOps, SecOps, *Ops, CI, CD, Containers, VMs, and all the other technologies we've devised over the past ten years, you always end up at the basic building blocks.

You eventually come to the conclusion that all we're really doing with all these new tools is adding software layers on top of those building blocks in an attempt to make them easier and faster to consume.

And how have we done overall?

Not bad, if you ask me. Some solutions are overkill for most people (K8s is an example of over kill for a start up and even an SME.) But Terraform, Ansible and GitLab (CI) are something I'm currently developing a highly opinionated video training course on because I believe they strike the right balance of improving on prior experiences without taking the absolute piss.

[_jwjg]

I am a developer, who also dealt with ops in a small business context. I agree with Ansible striking a good balance between prior experience and the future of automating server configuration.

I did a write-up on how I used it on my blog: [link redacted]

The workflow worked really well, provisioning Vagrant servers in staging and Digital Ocean droplets in production.

[GordonS]

Thanks, I appreciated this blog post - I've struggled to get started with Ansible before, and this was just what I needed!

[movedx]

Nice write up. Good job, mate.

I moved away from Vagrant in favour of Terraform, but I agree Vagrant still holds its own and is a great choice (HashiCorp really nailed it, eh?)

[high_5]

I agree with _dsr, he really nailed it. As for GitLab I agree they got the balance right except that it feels quite Kubernetes centric.

[cpb]

There is a lot of flexibility in the CI/CD configuration. You don't need to use the Kubernetes stuff.

[vageli]

Curious about where you see ansible fit in while using Terraform, could you expand on that? Everywhere I have thought I would need ansible to scratch an itch it has turned out that terraform has that functionality in some way (through null_resource, runners).

[movedx]

Terraform is for managing infrastructure. Ansible is for managing configuration. You can argue they're the same thing, but I disagree.

I believe in one tool to do one job really well.

Terraform is excellent at provisioning and managing infrastructure due in part to its DAG and HCL. On the other hand Ansible has been tuned over the years for managing configuration and the state of anything and everything from the OS upwards.

I also believe in using building blocks to get to where you're going, and these two bad boys click together quite well.

[vageli]

I guess what I'm questioning is the place of configuration management tools in a world of increasingly managed services where the server is not patched by you. In those cases, it makes no sense to me to patch individual containers through automation versus updating the image and pushing out the artifact to the service so all containers everywhere are updated and there's no checking for variance in state since all are running the same (updated) image.

[wokwokwok]

personal experience only, but like Dockerfiles, terraform is only good for provisioning until its not.

Once your VM or container hits a complexity point above trivial, ansible is very much a useful tool for provisioning container states, and specifically for patching container images to, eg. include security updates.

...beyond that, as in, the intended use case of dynamically updating multiple live machines in parallel... dunno, I don’t use ansible for that... but it beats the hell of out having a single monolithic batch script to setup a container. I use it for that purpose all of the time.

[vageli]

I guess that's the disconnect for me. Why would I want to update individual containers when I can just push out a new image and have automation rotate my services? Individually applying security patches at the container level also means there's probably SSH access as well, something I am quick to remove in environments in which I encounter it.

For host based security patches (if I'm in an environment where the servers aren't managed), adding an item to the crontab in user data usually handles that, and again any fleet-wide changes would usually be propagated by updating the user data, pushing out the change and having automation rotate the fleet.

[peterwwillis]

Just some minor clarifications:

- DevOps is a peer with Agile and Lean. Scrum and XP are Agile implementations. Scrum doesn't prescribe ways to code, XP does.

- 90% of what people develop or run today should be in containers, and not because containers are great, but because of the DevOps patterns of IaC, immutability, reproducibility, homogeneous environments. Whether you run them on your laptop, a VPC, AWS Fargate, a K8s cluster, etc is dependent on your business needs.

- Continuous Integration and Continuous Delivery aren't so much tools as a practice, and they're more complicated to implement at scale than just using a tool. There are some great books on the subject.

[StreamBright]

>> 90% of what people develop or run today should be in containers, and not because containers are great, but because of the DevOps patterns of IaC, immutability, reproducibility, homogeneous environments

Sorry but no. Container is __a__ way of achieving a small part of what you are talking about but not the only way.

Break it down:

- IaC: how do you containerise a load balancer? Terraform gives you infrastructure as code without containers.

- immutability: VMs, AMIs are immutable just like containers are (discounting the entropy that happens in every OS)

- reproducibility: Same, VMs, AMI, Terraform, Ansible all give your that

- homogeneous environments: Not sure what you mean by that, your Cisco or Juniper firewalls are not running in Docker so I am pretty sure you already have "heterogenous" environment if you meant that by what you wrote

I absolutely disagree this approach that we need containers for the reasons you just mentioned.

[peterwwillis]

> - IaC: how do you containerise a load balancer? Terraform gives you infrastructure as code without containers.

1) Terraform should be run in a container so that it will actually behave the way you expect, 2) containers are application environments built based on a Dockerfile, which makes it IaC.

> - immutability: VMs, AMIs are immutable just like containers are (discounting the entropy that happens in every OS)

True. But containers are easier and more portable, which is important to supporting the other aspects involved. Containers thus are a better general solution.

> - reproducibility: Same, VMs, AMI, Terraform, Ansible all give your that

Containers and VMs just... work. They're just collections of files. Very reproducible. Not 100% - you may need different guest drivers/kernels, different arguments to run your container in your particular system. But they're conceptually and operationally simple.

Terraform and Ansible are garbage fires of reproducibility and immutability. I could write a book on all the different ways these tools fail (most of it stemming from people trying to use them as interpreted programming languages, but also their designs are crap). There are whole frameworks built around Terraform and Ansible just to make sure they work right. They are overcomplicated, fragile bash scripts, and I'm quite frankly sick of using them. I think their entire existence is evidence of a huge gap in understanding how we should be operating systems today. [/rant]

> - homogeneous environments: Not sure what you mean by that [..] I am pretty sure you already have "heterogenous" environment

Those are opposites; homogeneous means "of uniform structure or composition throughout", heterogeneous means "consisting of dissimilar or diverse ingredients or constituents".

A homogeneous environment in a DevOps sense is when all environments have the same components and are operated the same way, and thus provide the closest results possible. This is incredibly important to prevent the classic "Well, it worked on my machine!" dev->production breakdown.

Homogeneous environments apply to lots of different things, but in the context of containers, they ensure that the environment the dev used to build the app is the same as what is in production. They also ensure that any scripts, tools, etc will use the same environment, if they are run in containers. I've wasted so much time in my career "correcting" heterogeneous environments in a bunch of different ways, whereas with containers the equivalent fix is "Please run the correct container version. Thanks"

The more systems you have, the more important this gets. At a certain point, the best choice is just to use baked VMs or containers for everything, everywhere, and containers are just so much easier, almost exclusively because Docker shoved so much extra useful functionality in. (I'll add that I do not necessarily like containers, but I do find them to be the most useful solution, because they solve the most problems in the most convenient ways)

[dsr_]

Please do me the favor of engaging with what I wrote.

[peterwwillis]

I thought I did? Also, this feels a bit passive-aggressive, did I do something wrong?

[dsr_]

It was meant to be polite. Under the guise of "clarifying" what I said, you completely contradicted it, without even doing me the courtesy of addressing my statements directly.

If it helps, my core point is:

DevOps is the name we give to two philosophical ideas. The first idea is that the tools and methods of software development can be used to improve our ability to do operations work. The second idea is that siloing people with operational skill away from people with development skill is a terrible practice.

Along the way, I specifically denounced the idea that DevOps is a single methodology, or that some tools are more DevOps than others, or that DevOps makes prescriptions about what you should do. Those are all things that you immediately advocated.

[peterwwillis]

I wasn't sure how much your ideas deviated from mine, which I why I said "clarifications"; but you're slightly incorrect. DevOps isn't two philosophical ideas. It's lots of things, and all those things are the methodology. There are many books, podcasts, blogs, conferences, etc that go over all of the things DevOps is. It actually has little to do with tools or software, even though that's basically what it was created around. It is a general methodology, and you pick how you implement it. You can even apply DevOps to non-software processes.

Look at it this way: The Toyota Production System isn't about cars. It was developed specifically to produce cars as well as they could be, but it doesn't address "car problems"; it addresses business problems, production problems, workflow problems. It applies methods as practices in ways that are specific to the production of cars, but you can apply the principles of TPS to things other than building cars (as we do with Lean).

DevOps is comparable to TPS (well, Lean), but for software instead of cars, and it borrows from other systems, and it has a few of its own ideas specific to software.

> Along the way, I specifically denounced the idea that DevOps is a single methodology, or that some tools are more DevOps than others, or that DevOps makes prescriptions about what you should do. Those are all things that you immediately advocated.

I advocated using containers because they help reinforce DevOps principles better than alternatives. You don't have to use them, but that doesn't make them un-applicable to DevOps. There are different levels to DevOps, and one of them is "practices": particular ways of doing things that DevOps encourages, such as Infrastructure as Code, Immutable Infrastructure, Heterogeneous Environments, Continuous Integration & Delivery, etc. Things that containers are more useful at accomplishing than, for example, VMs.

You don't have to use Kanban to run a car production line. But it's more TPS than the alternatives.

[lioeters]

I wish I could upvote this multiple times. Thanks for sharing your knowledge!

[nurettin]

After you accept that all these tools were built for overworked and stressed people who don't really have the time to learn things deeply, it becomes much easier. In fact, most of the programming ecosystem and systems administration works like this.

Try focusing on "what do I want?", get a superficial understanding on how the tool works, then try to apply that knowledge to your search engine query.

For example: Say you know that docker has images and containers. That means it is somehow going to install an operating system into your operating system and make an image. Then you will copy your program into that image. Then you will start a container (an instance) based on that image. And this is basically all you need to know about docker to start searching for how to do things. Like "how to build a docker image?" or "how to start a docker container?".

Another example. You know that Integration Testing means running your servers and running tests against them as if they are in actual production and continuous integration is a service that runs your integration tests everytime someone merges a branch to a monitored branch in a version control system. From here on, you are able to look up how to set a monitored branch, how to create a build machine and how to scale it.

[elevenoh]

This is how you end up with low-staying-value understanding built on shaky, fragile footing.

[owenmarshall]

"And what the heck do you call an act like that?"

"I call it 'modern software development practices!'"

[noxToken]

You can but not necessarily will. It can be valuable (to an extent) to abstract certain parts of process away from new users or employees to aid in bootstrapping. That's the message that I get from GP's comment. What these tools can't do is replace experts that can get between the commands to diagnose the root cause of an issue rather than the taking stabs at the dark of a cryptic error message.

These tools for DevOps are no different than the tools and tutorials for developers. It's fine to copy-paste a tutorial that launches an entire webapp from scratch just to get your project off the ground, but if you don't eventually learn what that 10-minute autorun.sh script is doing behind the scenes, you'll fall behind.

[aldoushuxley001]

Debatable, I think this sort of investigation is what leads to a more robust understanding compared to some abstract, non-applied information that has dubious relevance st the moment.

[nurettin]

This is how I get air humidity experts commenting on approaches to IT problems.

[orthoxerox]

What most enterprises think is DevOps is different from what dsr_ wrote.

They have admins that maintain their pool of servers.

They have developers that are fluent in the stack of their application.

They decide they need to have some of that cloud, containers, CI/CD stuff.

Turns out they need people who can write code that builds their programs, tests their programs, packages their programs, provisions cloud infrastructure, sets up that infrastructure, deploys the packaged program on their infrastructure and finally monitors its health and performance.

Most of their admins say they are not programmers, so it's not their job.

Most of their programmers say they are there to write Java/C#/Python/JS, so it's not their job either.

They find some people who don't mind learning all these things and call them their DevOps team.

In a not perfect, but generally just world this team disseminates their knowledge across both programmers and admins, making both aware of each other. Programmers now think about the infrastructure they need, admins now think about the workloads their infrastructure runs.

In an unjust world, you end up with three silos. Programmers say their code compiles on their machine, admins say they have installed the new server, devops frantically try to build some pipeline that deploys that code on that server.

[StreamBright]

>> What most enterprises think

You mean the companies we regularly fail every aspect of engineering?

dsr_ summed up pretty well how Amazon and a bunch of companies think abut DevOps. Coincidentally these companies produce the highest grade of software, tools, services etc.

>> Most of their admins say they are not programmers, so it's not their job.

What you are describing is the 90s approach to IT. These companies disappear really fast. IT is changing just like agriculture was changing long time ago. Toffler talks about this in The Third Wave.

Old approach: lets do everything by hand New approach: automate most of the things you can

>> In an unjust world, you end up with three silos. Programmers say their code compiles on their machine, admins say they have installed the new server, devops frantically try to build some pipeline that deploys that code on that server.

I migrated countless companies from 90s approach to CI/CD world, they never looked back. You just think that because there are late adopters this world is going to exist indefinitely. I do not think so.

[orthoxerox]

I was not talking about IT companies that produce h/w or s/w goods and services. I was talking about companies that produce other kinds of goods and services and insource or outsource IT services that support their value chain. They don't necessarily feel the same pressure to improve. I agree they are late adopters, but late adopters aren't stragglers. There will still be enough of them in 10 years time.

[movedx]

I'm currently in the process of developing a video training course that teaches Terraform, Ansible, Packer and GitLab (CI) as a set of interwoven, dependent tools. Is this something you feel would scratch your itch?

Would you be willing to have a quick chat? I'd pay for your time, of course. I need to gather feed back from people looking to develop their skills into the CloudOps space and understand what it is they're looking for.

[programmertote]

I have been looking for a entry point to start learning containerization and other fancy, related tools that I've been reading on HN in the past couple of years. I mostly write backend code/scripts, so I'm very new to dev ops stuff.

The immediate need for me is to package up a C#/.NET web app and its components (DB, etc.) into a container so that I can deploy it on any big-name cloud provider (Azure, AWS, Google Cloud). Now after reading through the comments in this HN post, I am not sure if I should choose Docker or something else. If you have any suggestion, I would love to learn. I'm more than happy to provide you with feedback and such (even for free) if I can learn from your tutorials. Thank you.

[movedx]

I think I would start by looking at how-to craft that container by hand. You'll understand the fundamentals better if you do it that way. Once you have a grasp of those, research the tools that then do it for you.

If you deploy a container to a Cloud provider, you'll need to first setup and understand the container engine as well. Not a bad thing to learn, for sure, but again start small and from the bottom upwards.

My tutorials won't cover containerisation because frankly I believe they're overkill for most situations. Sure they're fast and so on, but a slower golden image and a simple EC2 Instance in an AutoScaling Group is easier to understand, manage, and can be just as easily orchestrated.

[aeorgnoieang]

I'd suggest starting with just scripts, e.g. Bash, PowerShell, anything – that's assuming you have automated builds for both your web app and its components first. (So, automate builds of your app and all of its components first if you don't already have that.)

Pick one cloud provider first and write some setup scripts, i.e. scripts to build an initial (minimal) environment, 'from scratch', for your app and its components, e.g. create a new EC2 instance for the web app, upload your build package to it, etc.. Write the scripts so that any 'secrets', e.g. your AWS API key, are provided as either environment variables or regular command line arguments.

Then, still for the first cloud provider, write some update scripts, i.e. scripts to update your web app and its components.

Assuming your app, or its deployed 'instances', are relatively small and intended to serve a modest load, I'd suggest starting out treating the cloud servers or services as 'pets', i.e. entities you distinguish by name and for which you would be 'sad' if they 'died' (crashed or shutdown). At larger scales, it's often worth treating servers as 'cattle', i.e. a mass of nameless entities, but you probably won't need that at this point or anytime soon. (You'll know better tho.)

As for "containerization and other fancy, related tools that I've been reading on HN in the past couple of years", they're just like any other software – tools that can be used but aren't ever strictly necessary.

Containers are, basically, virtualized OSes, and they can be (very) useful. In my opinion, they're most useful as a way to bundle components of an app with an OS (and other OS components). That you can run those containers in different environments and be reasonably assured they're (mostly) identical can be a big benefit. But there are associated costs too (as with anything)! But if your app and all of its components can comfortably fit on a single (virtual) server, the cost of any changes you need to make to your app to run inside them are probably not worth the (currently) modest benefits.

And all of the other "fancy ... tools" are generally even more a matter of tradeoffs you'll need to make. Once your app, or (production) instances of it, are distributed over several, or many, servers, and you start adding things like load balancers, caching, separate search services, etc., then the benefits of the other 'fancy' tools will start to make more sense.

But, like with many things, it's good experience to directly run into some of the issues that containers and the other fancy tools aim to solve, and really try to solve them yourself with DIY solutions, before committing to use yet another program or tool to do it for you.

Of course, if you just want to learn those tools, and you'd like to use your own app as a 'motivate example', that's perfectly reasonable and valid too. I would recommend tho not to lean on those tools for your own immediate needs unless you really need them.

[swyx]

nice guide to devops linked at the bottom of the article https://www.techrepublic.com/article/devops-the-smart-person...

[marsrover]

I think the only way to learn them is to use them. You can do that locally or purchase a VPS and work there.