[zeroCalories]

It's not clear to me that being completely unaware of your infrastructure is a good thing. I don't think it's too much trouble to ask an engineer to understand k8s and think about where their service will live, even if it's a ci system that actually deploys. Furthermore, many layers of abstraction, especially in-house abstraction, just mean you have more code to maintain, another system for people to learn, and existing knowledge that you can't leverage anymore.

[opportune]

There is a wide spectrum of infrastructure (and platforms, frameworks, etc) from “allows applications to do just about anything, though it may be very complex” to “severely constrains applications but greatly simplifies doing things within those constraints.” To be clear by “just about anything” I am not talking about whether some business logic is expressable, but whether you can eg use EBPF and cgroups, use some esoteric network protocol, run a stateful service that pulls from a queue, issue any network call to anything on the Internet, etc.

If you are developing applications software like Uber 99.99% of the time you really do not need to be doing anything “fancy” or “exotic” in your service. Your service receives data, does some stuff with it (connects to a db or issues calls to other services), returns data. If you let those 0.01% of the things dictate where your internal platform falls on that spectrum, you will make things much more complicated and difficult for 99.99% of the other stuff. Those are where leaky abstractions and bugs come from, both from the platform trying to be more general than it needs to be and from pushing poorly understand boilerplate tasks (like configuring auth, certifications, TLS manually for each service) to infrastructure users.

Being unaware (of course not completely unaware, but essentially not needing to actively consider it while doing things) of infrastructure is actually the ideal state, provided that lack of awareness is because “it just works so well it doesn’t need to be considered”. It means that it lets people get shit done without pushing configuration and leaky abstractions onto them.

I’ll give you one example of something that does an excellent job of this: Linux. Application memory in linux requires some very complex work under the hood, but it has decent default configurations with only a couple commonly changed parameters that most applications don’t need much, and it had a very simple API for applications to interface with. Similar with send/receive syscalls and the use of files for I/O ranging from remote networking to IPC to local disk. These are wonderful APIs and abstractions that simplify very hard problems. The problem with in-house abstraction isn’t that they are trying to do abstractions but that sometimes they just don’t do a good job or churn through them faster than it takes them to stabilize.

[intelVISA]

Well put, 99% of companies don't need to introduce such complexity for their relatively trivial use cases (though well-intentioned albeit bad engineers will try to invent it anyway).

[jbotdev]

Part of my point is the goal with such a system is usually to require less infra work/knowledge from your devs, but it backfires if you don’t invest enough in your abstraction.

The implicit goal of these abstractions is really to central knowledge and best practices around the underlying tech. Kubernetes itself is trying to free developers from understanding server management, but you could argue it’s not worth using directly vs. just teaching your devs how to manage VMs for the vast majority of organizations.

I don’t think you’re ever going to stop more and more layers of abstraction, so the best we can hope for is they’re done well. Otherwise you may as well go back to writing raw ethernet frames in assembly on bare metal.

[zeroCalories]

> Part of my point is the goal with such a system is usually to require less infra work/knowledge from your devs, but it backfires if you don’t invest enough in your abstraction.

I disagree that the solution is to simply build more. Often the best thing to do is accept that devs will need to know a little infra, and work with that assumption.

> The implicit goal of these abstractions is really to central knowledge and best practices around the underlying tech.

I agree with that.

> Kubernetes itself is trying to free developers from understanding server management, but you could argue it’s not worth using directly vs. just teaching your devs how to manage VMs for the vast majority of organizations.

The difference is that spinning up a VM and setting it up to have all the features you would want from k8s would be too much to ask from a dev. You would probably just end up re-creating k8s.

> I don’t think you’re ever going to stop more and more layers of abstraction, so the best we can hope for is they’re done well. Otherwise you may as well go back to writing raw ethernet frames in assembly on bare metal.

The problem is that abstractions are not free, and most of the time they aren't done well. Once in a while you'll get one that reduces(hides) complexity and becomes an industry standard, making it a no-brainer to adopt, but most of your in-house abstractions are just going to make your life worse.

[pm90]

I think the biggest “win” with abstractions is that it makes it easier for infra teams to update underlying concretions (is that a word? the concrete version of the abstraction) without having to dig deep into the codebase.

e.g. with kubernetes, if you have the actual manifests defined by every team, it is a pain to do any sort of k8s updates. With a simple abstraction where teams only define the things they are interested in configuring (eg helm values), that simplifies this task a lot.

[oceanplexian]

All it takes is for one microservice to start hanging on a GRPC request, server hardware stops doing some fundamental thing correctly, or some weird network quirk that 10x’s latency to half the switch ports in a rack, and you end up with insane, sophisticated cascade failures.

Because engineers don’t have to understand infra, it often spans geographies and failure domains in unanticipated, undetectable ways. In my opinion the only antidote is a thorough understanding of your stack down to the metal it’s running on.

[scarface_74]

A single engineer can’t understand everything at scale.

Even in a 100 person startup that I worked for where I designed the infrastructure and the best practices and wrote the initial proof of concept code and best practices for about 15 microservices it got to the point where I couldn’t understand everything and had to hire people to separate out the responsibilities.

We sold access to micro services to large health care organizations for their websites and mobile app's. We aggregated publicly available data on providers like licenses, education etc.

Our scaling stood up as we added clients that could increase demand by 20% overnight and when a little worldwide pandemic happened in 2020 causing our traffic to spike