[hsn915]

Think about it from the other side:

Dealing with the infrastructure is your job. You know the ins and outs. It doesn't seem complicated to you.

For someone who is dealing with the application logic, having to context switch to infrastructure is painful. I have no idea how you have set things up. When I read your documentation I'm just baffled at the sheer amount of complexity. I don't want to deal with it. It's not my full time job. I have other things to worry about. I have no mental space to deal with this headache. I will just ask you so I can move on to do my actual job - which is already taking up over 90% of my mental capacity.

When I setup my own infrastructure, I make it as simple and stupid as possible, precisely because I have no mental slack to deal with all the complexity that I see devops people deal with on a daily basis.

At most jobs I've been to, the complexity I've seen in infrastructure is mind boggling. My intuition is that this complexity is not actually needed if you remove all the cruft and think from first principles about what the actual problem is you are trying to solve.

However, people in devops seem to enjoy building and maintaining this kind of complexity, and even take pride in how everything is orchestrated.

To me it's just a headache.

[jreese]

The drain I am talking about is a complete lack of due diligence on behalf of those asking for my help (and my limited time/attention).

When the error message contains a clear message and a URL with step-by-step instructions, and the person in question just pastes the full error message including the URL, and asks "what do I do?"

When a problem can be solved by responding with "please follow the instructions in the error message you sent me".

When someone makes an internal group post asking "how do I do X?" and the pinned post that was right below the submit button contains a summary and link to detailed step-by-step instructions titled "how to X".

[draugadrotten]

I've been in your shoes and I've accepted this fate.

I am the expert. I know things. I choose to change my viewpoint to taking pride in making the best possible explanation that serves the users' need. When they ask me "what do I do?" I no longer snicker or sigh that they don't know or that they are too lazy to find out. Instead I copy-paste the answer they need. It takes me seconds, perhaps it saves them thinking. Thinking seems hard to some people. I choose to believe I save the company some time when I think for them.

Yes, they do not learn when they are spoon-fed the answers. Yes, they could have figured it out. So what if they didn't? They are producing whatever they are supposed to produce.

When I get too tired of the same questions, I change jobs. I am in a C-level position now, and somehow the questions from other C-level managers are similar. They have no idea how they should deal with things from my business area, and that's okey. I'm here to help. I help. When I get too tired of the same questions, I will change jobs.

[hsn915]

The way I've seen things work is like this:

devops sets up something convoluted that they for some reason like.

Developers are confused by the thing. It causes them endless problems.

Documentation is sparse, or out of date, or just plain wrong. The instructions to fix problems omit key requirements because the person who wrote them just assumed the reader knows about X and Y when the reader often has no clue. You follow the instructions and the problem is not solved, and maybe to make matters worse, you run into new problems.

The way I see it is: if developers enjoyed dealing with infrastructure, devops jobs would not exist, it would just be something that developers do as part of their job.

As a devops engineer, you should think of your fellow software engineers as your users, not as your "project members".

Your job is to make it as painless and problem-free for them as possible. If problems arise constantly, that's not the user's fault; that's your fault.

[omegalulw]

> My intuition is that this complexity is not actually needed if you remove all the cruft and think from first principles about what the actual problem is you are trying to solve.

Have you ever worked on infrastructure? This comment is wrong on soo many levels.

[Aeolun]

> Have you ever worked on infrastructure?

Not really. From an application perspective you can generally get away with two really big servers (one of which is the backup), and a load balancer.

There may be more involved in the surrounding infra, but that’s stuff the application developer will never have to touch.

[hsn915]

This comment is not wrong on any level.

I have worked in many aspects of web development. I know for sure that everything in the process is way too complicated for what it does. Unnecessarily so.

There's no reason to think that infrastructure is the one place where all the complexity is there for good reason.

Have I worked on infrastructure? Yes and no. Depends on what you mean.

I have worked on infrastructure in the sense that I have setup the infrastructure for websites that can handle thousands of concurrent requests, and the infrasturcure I setup is stupidly simple, everyone who sees thinks there must be something missing. But there isn't anything missing. It handles the job much better than the complicated mess that I see in all other companies.

Now, the thing I haven't done is actually setup the infrastructure for a truely distributed system that is horizontally scaled. I do all my scaling vertically because computers are so fast and storage capacity is so large that one server can handle thousands of concurrent users without nearly sweating.

Part of the reason everyone else has a super complicated infrastructure is because they chose bad software technologies to develop their applications, such as, python.

Yea, if your application backend is in python or ruby, you have no choice but to create a complicated infrastructure. Because those languages are so slow, you just cannot scale vertically. That's not an option. You get an explosion of complexity because you also need additional servers to handle stuff that would normally just be part of the application code.

For example, when I program in Go, and I need to cache some data, I just create a map, or something, to cache data in memory. But if you have Ruby, you can't just do that! Since you have 200 application instances running on aws, you want to use something like redis to act like a shared distributed cache. So now your infrastructure has to include information about how the application servers need to communicate with the caching servers.

All of this is unnecessary complication that can go away if you stop what you are doing and just analyze the problem from first principles.

Your massive convoluted infrastrucutre is only there to compensate for the bad decision you made early in the process, which was to use a slow interpreted language to write the backend of your system.

Most of the time the real solution to scaling problem is just writing better (and simpler) code.

I've seen at more than one company, how this complciated infrastructure does nothing to make the application robust because it grinds to a halt when there are about 1000 concurrent users.

[omegalulw]

I guess there is a disconnect between what you mean by the term infrastructure and what I mean.

Let's take an example, say Reddit. You want to support iOS, android and web. All platforms have the same core data but maybe have different APIs for optimizing queries, e.g. you may want to use server side rendering for web.

So here's some of the microservices that you will use server side. I will label which I consider to be infra:

LoginService (infra)

Database server or library (to create consistent views/transactions on top of your raw database, infra)

Core business logic server

Web frontend server

Android/iOS frontend servers (probably not needed)

Image/audio/video processing (infra)

ML server (infra)

I don't see how any of the infra services here are trivial.

[hsn915]

If by infrastructure you mean microservices, then it's even worse.

I wholly reject "microservices", the philosophy and ideas behind it, the tooling and the culture surrounding it.

It's not only too complicated. It's worse than useless. It's extra work that produces negative value.

[adminscoffee]

there is a saying, you may think you are intelligent but you are only as intelligent as what your student is able to learn from you.

it's basically "beautiful mind" syndrome, and it's steeped deeply in ego.

now if on the other hand that teacher is coming from a place where it was hard for him, he probably has a sense of empathy and is willing to teach.

[baash05]

Totally agree.. As a rails dev I'm asked to dig into ops.. Not once is the JS team asked to build out the rails code, or fix things in it when it glitches. The api business logic falls down, and thats on me.

But when ops is glitchy, I have to blow half my day faking interest in docker? I mean best case, I learn how ops works, and then what. I become the one they tap when ops is swamped, then before I know it, I'm doing ops.

And I quit, because as much as I love the ops team, I hate the gig. I'd hate to be a designer, or a BA. It's not where I find joy.

So when I hear people say.. Just follow these steps to trouble shoot, I wonder if it would be cool to tell that to our users..

[therealdrag0]

Think about it from the other side:

Infrastructure is like application dev in that it’s a bundle of histories of feature work over time and with many contributors, and many customers with competing wants and not enough time to satisfy everyone.

Granted a primary goal should be to have simple APIs/workflows for app devs to use, but complicated insides is as natural as any advanced system.