[omegalulw]

> My intuition is that this complexity is not actually needed if you remove all the cruft and think from first principles about what the actual problem is you are trying to solve.

Have you ever worked on infrastructure? This comment is wrong on soo many levels.

[Aeolun]

> Have you ever worked on infrastructure?

Not really. From an application perspective you can generally get away with two really big servers (one of which is the backup), and a load balancer.

There may be more involved in the surrounding infra, but that’s stuff the application developer will never have to touch.

[hsn915]

This comment is not wrong on any level.

I have worked in many aspects of web development. I know for sure that everything in the process is way too complicated for what it does. Unnecessarily so.

There's no reason to think that infrastructure is the one place where all the complexity is there for good reason.

Have I worked on infrastructure? Yes and no. Depends on what you mean.

I have worked on infrastructure in the sense that I have setup the infrastructure for websites that can handle thousands of concurrent requests, and the infrasturcure I setup is stupidly simple, everyone who sees thinks there must be something missing. But there isn't anything missing. It handles the job much better than the complicated mess that I see in all other companies.

Now, the thing I haven't done is actually setup the infrastructure for a truely distributed system that is horizontally scaled. I do all my scaling vertically because computers are so fast and storage capacity is so large that one server can handle thousands of concurrent users without nearly sweating.

Part of the reason everyone else has a super complicated infrastructure is because they chose bad software technologies to develop their applications, such as, python.

Yea, if your application backend is in python or ruby, you have no choice but to create a complicated infrastructure. Because those languages are so slow, you just cannot scale vertically. That's not an option. You get an explosion of complexity because you also need additional servers to handle stuff that would normally just be part of the application code.

For example, when I program in Go, and I need to cache some data, I just create a map, or something, to cache data in memory. But if you have Ruby, you can't just do that! Since you have 200 application instances running on aws, you want to use something like redis to act like a shared distributed cache. So now your infrastructure has to include information about how the application servers need to communicate with the caching servers.

All of this is unnecessary complication that can go away if you stop what you are doing and just analyze the problem from first principles.

Your massive convoluted infrastrucutre is only there to compensate for the bad decision you made early in the process, which was to use a slow interpreted language to write the backend of your system.

Most of the time the real solution to scaling problem is just writing better (and simpler) code.

I've seen at more than one company, how this complciated infrastructure does nothing to make the application robust because it grinds to a halt when there are about 1000 concurrent users.

[omegalulw]

I guess there is a disconnect between what you mean by the term infrastructure and what I mean.

Let's take an example, say Reddit. You want to support iOS, android and web. All platforms have the same core data but maybe have different APIs for optimizing queries, e.g. you may want to use server side rendering for web.

So here's some of the microservices that you will use server side. I will label which I consider to be infra:

LoginService (infra)

Database server or library (to create consistent views/transactions on top of your raw database, infra)

Core business logic server

Web frontend server

Android/iOS frontend servers (probably not needed)

Image/audio/video processing (infra)

ML server (infra)

I don't see how any of the infra services here are trivial.

[hsn915]

If by infrastructure you mean microservices, then it's even worse.

I wholly reject "microservices", the philosophy and ideas behind it, the tooling and the culture surrounding it.

It's not only too complicated. It's worse than useless. It's extra work that produces negative value.