The view in this article is that DevOps is a role, it was never meant to be this way.
The whole purpose of DevOps was to break down barriers between Dev and Ops, by not creating specialist roles and allowing developers to take ownership of ops, removing the need for the SysAdmin gates.
Instead what this article highlights is that we’ve moved the complexity to specialist tools which require another specialist role.
I would say this suggests you’re solving the wrong problem.
I’ve never really gotten this argument. People can go really deep on Terraform, Cloud Automation, CI/CD, containerisation etc without being developers or sysadmins.
“DevOps Engineer” fits that role perfectly in spite of the breaking-down-siloes goodness.
Then you're doing it wrong. My teams have two components that require review from a specific group:
1. Core infra
2. Anything related to security
Outside of those, we all can and do work on everything else. We regularly coach people in terraform and third party APIs to endure they understand that they own their application lifecycle. They control the entire SDLC and we work together to ensure sanity. There are lots of bumps but nobody gets a pass on understanding the in and outs of software development and management. I can't see a cloud based offering being successful without that.
I have been hearing for at least 10 years that the role of sysadmin is dead....
Yet they are in more demand than ever today.. The role has changed, sysadmins today need to know more things that were exclusive to "devs" in the past, like version control... but administering systems weather they are fields of cloud cattle or onprem pet servers I still do not see the role of sysadmin going away...
sysadmins may manage more servers per admin, they may have other roles in addition to what as traditional sysadmin 15 years ago, but the vision of the author of a magical system were devs just hand over their code to AI, I think it fantasy
I think you're mixing theory with practice; DevOps should not be a role, but it often is...
As for requiring another specialist role, that's not the goal of building an automated system. Or at least, this specialist will not part of the team that's building the product, but rather of the company that's building this automated platform. If you can automate the orchestration and management of Kubernetes or serverless, then developers just need to know how to build a container or upload code on a serverless platform.
We reassigned time. It’s not like you work 50hrs instead of 40.
At the time of Agile I used to hear managers say it’s a way to stress the developer more and make him work longer. No. Agile a way to relieve stress and improve adequation using tighter feedback loops.
Same for DevOps. DevOps is a way to close the feedback loop so developers don’t just throw the binaries over the fence and let sysadmins deal with stacktraces (by submitting a Jira ticket obviously). You’ll then be incentivized into developing production-proof software and prove it by supporting it; while you have other colleagues developing other features.
This post doesn't really say anything and hand waves over a lot of the complexities of real world deployment and operations that isn't a simple stateless hello world web application.
I don't think the point of the article was to say technically how to automate everything. But rather that the tools to automate are coming and that we should be on the lookout and ready when they come.
Multi-tenant, blue green deploys, infrastructure as code..
There are layers and layers of new tools and vendor specific infra required for the cloud. Previously I just needed a server, a git repo and a jenkins job.. now I need 4 services with different apis, roles, rights, a jenkins job and 3 different templating languages like terraform or ansible.
No software engineer is going to want to do ALL of that, and if you do.. you should be scratching your head if you're really developing actually business value rather than castles in the sky. Its nice to have new toys.. but if customers are still calling your colleages that crucial things like invoicing or ordering is still broken.. than there is no way to justify weeks of research and development on ops tools before solving athe actual problem ( which would just take hours previously.)
So yeah, we actually need more devops people if we really want to keep those new services so that software engineers can actually keep engineering software itself, else everything slows down.
Its a very poorly written article with all the buzzwords one can think of. DevOps is not even remotely close to dead, but rather evolved a lot. Gone are the days, when one could easily ssh into remote prod systems and fix the issue. With shell less Docker images and container orchestration tools (hashicorp stack), CI/CD pipelines, blue-green deployments etc, Devops has become a lot more sophisticated than before.
> Gone are the days, when one could easily ssh into remote prod systems and fix the issue. With shell less Docker...
I think we finally realized that fixing something in prod via ssh is not a good solution and might introduce new bugs on its own. Rather build an infrastructure that allows you to rollback fast. It is also not worth to fix individual machines in a big cluster, just throw them away and bootstrap them from zero. This way you make sure that you don't have accumulated patches and workarounds on your nodes that might lead to future failures. In some companies we reached the point where you don't even fix a cluster in a multi-cluster setup, but throw away the whole damn thing and bootstrap it from zero.
You say sophisticated but in reality, it has become extreamly complex, which is a natural consequence of using tons of different tools and glueing them together.
And that's why there's room for automation to take a lot of the complexity away.
Granted it's not going to be easy to have a platform where every tool can seamlessly connect, but we can start with baby steps such as services that automate multi-cloud deployments like the article suggests.
We are getting more and more servers, more containers, more complex, dynamic infrastructure, more API driven infrastructure etc. All of this is driven with automation created by DevOps engineers. In addition, there is an absolute mountain of old stuff to modernise and automate.
No. DevOps will stay and there will be SREs as well and there will be a hybrid of this combination as well
DevOps actually didn't break the barriers at all. It just created more roles inbetween Developers, testers and operations and brought in more tools. So people in operations are asked to use the Devops tools as organization has already bought it.
I waited and learned it for a bit but nobody seems to be raising salaries for me to know that crap in addition to other full-stack stuff, so NO, there are DevOps engineers and they get good money for me not to deal with that garbage for essentially free.
Aye, none of the devs on my team want to deal with infrastructure - they want to push their code and have it go to production. I deal with all that garbage so they don’t have to.
I think the title “DevOps Engineer” is a bit of a misnomer, though; my team as a whole does DevOps, I build and maintain the platform.
I think team members do varying amount of DevOps but everyone should be involved. When writing code thinking about observability and monitoring, watching deploys as they go out, checking back on things randomly or intermittently whenever there's an inkling to do so, etc.
So.. serverless apps will not require managing tons of vms and containers, so devops workload reduces 80%? Am I comprehending point? And prognosis is serverless apps will dominate?
I mean, in my small world of using azure app services, azure functions and CI/CD, that makes sense. I work on relatively small and tailored solutions though.
I'm in the process of migrating a handful of small internal systems from azure to AWS - azure is miles ahead in this regard so you're likely looking at it from the perspective of someone using the best tooling available. Setting up Azure Container Instances, or even Azure webapp compared to fargate/elastic beanstalk is just night and day
i think there's a clash of definition in that thread. Devops originaly was a job title for sysadmins with high automation skills and able to deploy to a cloud.
it later evolved to something much more management related.
Depends who you ask. Everyone seems to invent the definition that local-optimize for themselves. The book Good To Great goes into the dynamics, outside of IT, so is a general pattern.
It was, but in the end it couldn't, because as it turns out "developers" aren't really programmers and even the plan to make them responsible for their own messes is ultimately failing... very, very few people are truly meant and talented to program a computer, the rest have no business being in this industry and are in it for all the wrong reasons.
For programming a computer, correctly, elegantly, has always been an elitist, and will forever remain an elitist thing to do. It requires patience, dedication, perfectionism, something most of the people currently "developing" do not and will never have.
I have been preaching automation on the level of systems exchanging messages and reaching decisions based on the exchange of those messages for decades, which is why my designs have stood in production for decades. What will replace DevOps? OS packaging, as it was meant to, because it is the correct, optimal solution to very large scale automation of autonomous systems, which require no human interaction or oversight.
This is a garbage quality post but the underlying premise I think is fair. I’ve heard much more reputable people like Kelsey Hightower make similar claims recently.
Just as a matter of personal interest I spent the last year looking into the question of how close could a small and possibly even competent one person team get to doing “cloud native” “best practices” from day one if they made the right choices.
My assessment is that, we as an industry are right now, entering an interesting stage where this is starting to look vaguely possible thanks to some recent offerings.
Running Kubernetes for example has in just the past couple of months potentially become a genuinely hands off exercise thanks to GKE Autopilot where you can just throw containers at it and it will just work and do the right thing by default.
Granted this is only one part of the problem however.
The other super powerful component of K8s however is it’s idea of a reconciliation loop. People are starting to see this as not JUST a tool for managing application lifecycles and scaling but now as one of the key new primitives in automation.
If you can specify:
1. The current state of a system.
2. What code needs to be run in order to transition from one state of a system to another
Then you can create a custom resource definition in order to manage that for you. This is where you’re starting to see a lot of interesting new things emerge. For example while tools like Terraform are great to setup infrastructure they don’t have a solution for say when someone starts manually adding, removing or otherwise configuring your infrastructure outside of it. That’s a great example of a problem that can now just go away and this is where you’re seeing a new generation of tools emerge in that space like Crossplane.
Now I can not only specify my infrastructure as code, have everything go through a gitops style workflow but I can also rest assured that I won’t ever have a configuration drift problem again.
Other cool areas I’ve noticed emerging at the application development level include projects like Dapr which is one of the best things to come out of Microsoft in years I think. But for those who are unfamiliar with it their promise is again built on top of Kubernetes custom resource definitions and says here is a single standard API which you can use and we will do pretty much any task relevant to distributed applications for you using best practices and you never have to think about it or touch it. So this means I as a developer or an operator am no longer thinking about things like service discovery or having to set up read replica databases anymore etc.
Finally, one more piece of the puzzle I’m excited about is a project called “open application model” which once again is built on top of custom resource definitions and let’s me as a developer to just specify my various applications as common components and traits I.e this is a service and it should be available to the outside world.
With that information it has everything it needs to once again just go and do the right thing and I never need to think about it again.
Basically, I agree with what is said here even if it’s poorly written. I don’t think right now is necessarily a great time to go and become a Kubernetes expert because the need for them is likely to become much smaller in the future as organisations instead move to various standards that CAN be automated in predictable and reliable ways.
P.S Open policy agent is another powerful abstraction that plays a key part here that I haven’t mentioned. But this is a great place to put all of the logic that defines “what is the right thing to do (especially if it is specific to my business)” and the rest of the tools I’ve mentioned here can use that. But normally coding this stuff manually at scale across distributed systems would have become a nightmare.
> Running Kubernetes for example has in just the past couple of months potentially become a genuinely hands off exercise thanks to GKE Autopilot where you can just throw containers at it and it will just work and do the right thing by default.
I never heard of Autopilot before but it sounds like what AWS has had for a while with Fargate where you don't need to manage your worker nodes, instead you define what compute resources you want and what tier of instances they should run on (CPU optimized, general, etc.).
Things like this are definitely a very welcome change but running a production workload on Kubernetes is still no where near a hands off exercise, even with the cluster creation and scaling aspect abstracted away. There's still learning, using and applying Kubernetes.
Sure once you have the knowledge and experience you might only need something like 200-250 lines of YAML to run a production grade stateless web app service on your cluster including a few bells and whistles like external-dns but getting there isn't hands off and things are changing, then there's also figuring out how to automate all of this into a CI / CD pipeline and ensuring any services being built are architected to run in a load balanced container friendly way (using object storage, generally stateless, env variables, etc.). Not all of these things are a given.
I just mostly went through this path in the last few months while working part time for a client. I'm basically at the point of wrapping things up with external-dns and it's taken around 80 hours to get there. That's not including automating all of this in CI yet too. In end there's no way I would classify myself as an expert either, there's still a lot left to learn, but that timeframe includes learning Kubernetes from scratch after already having worked with Docker / Ansible for 6+ years and being comfortable with sysadmin / ops tasks in general. It wasn't the hardest thing in the world but it wasn't anywhere near hands off. It was like learning any other complex piece of software. A crazy amount of trial and error while reading the docs and scouring the internet for as many resources as you can find.
The same process happened with Serverless too. It took a different amount of time to get a function running in a production ready way but there were a ton of things to learn along the way.
In both cases this required spending a lot of time learning things that a lot of developers either don't care about or don't have time to learn because they are busy developing business features. IMO the demand for ops style roles will never go away, even going with the highest level of abstraction with Heroku requires spending time setting it all up and keeping things in check.
I think maybe I didn’t do a good job explaining the various abstractions and how they work together but in the scenario I outlined above combining those things together you most certainly are not going to be handwriting K8s yaml files and I don’t think that’s going to be a common thing to do in the future let alone having to configure DNS yourself etc.
These are all very much things that can be and are in the process on being abstracted away.
We are actually fairly close to a point now where you as an application developer can just describe your apps and how they work and be basically done.
Knowing how to use kubernetes is not going to be any kind of a requirement in the not too distant future.
> We are actually fairly close to a point now where you as an application developer can just describe your apps and how they work and be basically done.
I don't know, DO launched their application deployment platform a few months ago and it's an abstraction on top of Kubernetes.
It's still in its early stages and while I use and love DO I didn't find this app platform to be near ready to run a complex web application on. It was really nice for running a hello world example web server and it sure looked great on a few demos videos but as soon as I tried to hook up a real application with a few moving parts I got pretty stuck to the point where I was hard blocked by it.
I'm sure DO's app platform will improve over time but people and organizations have applications to deploy today. Waiting around until the next thing is around, fully baked and battle tested isn't an option for most. It's almost always worth learning something fairly stable and implementing it and then incrementally changing it over time. If you always wait around for the new hotness you'll never ship anything and when you do, you'll always be in the most dangerous situation of being patient zero with limited documentations and bugs.
Heroku for what it is, does a good job but they also have 14+ years of experience and likely hundreds of thousands of dev hours behind it to create its abstractions and iron out the kinks, and despite that it's still not something everyone flocks to without question.
I think we have a very very long time to go before a serious contending solution comes along that perfectly abstracts away app deployment at scale to specifying a few configuration properties.
>Things like this are definitely a very welcome change but running a production workload on Kubernetes is still no where near a hands off exercise, even with the cluster creation and scaling aspect abstracted away. There's still learning, using and applying Kubernetes.
I think that's what this article alludes to. You won't need to manage Kubernetes with an automated container orchestration platform that just needs your container or cluster. If an automated system can take care of the management, then the "cloud management" part of DevOps is greatly reduced.
Comment makes me think real names are required for credibility. In these times posting as a moniker is more important than ever given the low bar for termination in today’s culture.
- DevOps originally was supposed to eliminate Operations (Sysadmin) staff by involving Dev with the Infra and Release process.
- But Dev is incentivized to create features, not know the intricacies of Release at scale, so dedicated Devops (Operations) teams were again staffed
- dedicated DevOps teams are here to stay except in certain unique scenarios. For example, I've seen some Indian-led SV startups that hate Operations staff ("oh, they're overhead") carefully design their production environment around using services that don't need mgmt. (DynamoDB, Lambda, etc.), and that will work up to a certain scale.
However, every time they have a problem with AWS, onboarding, security, excessive on-call rotations, etc. their developers have to be ready to become the DevOps they so despise, reducing the number of featueres they can build.
- in larger companies, Devs don't have access to production, so again dedicated DevOps teams are needed if only to enforce policy.
This aligns with some of my experience. Btw, your comment appeared 'dead' to me which didn't make sense... I learned about the ability to 'vouch' for the first time. Great feature. HN truly is a well evolved platform. Kudos to all roles for providing this service, visited and valued by so many wonderful geeks. Ty!
> in larger companies, Devs don't have access to production, so again dedicated DevOps teams are needed if only to enforce policy
Nor should developers be let near productions as they'll always try to root cause problems in production. Developers are their own worst enemies which is why DevOps is a misnomer.
> DevOps originally was supposed to eliminate Operations (Sysadmin) staff
It wasn't. It was supposed to break down the model of Dev and Ops being separate silos that don't work together well and don't understand/respect each others concerns.
The view in this article is that DevOps is a role, it was never meant to be this way.
The whole purpose of DevOps was to break down barriers between Dev and Ops, by not creating specialist roles and allowing developers to take ownership of ops, removing the need for the SysAdmin gates.
Instead what this article highlights is that we’ve moved the complexity to specialist tools which require another specialist role.
I would say this suggests you’re solving the wrong problem.