[dajonker]

I totally agree, but that's not what happens in reality: the average devops knows k8s and will slap it onto anything they see (if only so they can put in on their resume). The average manager hears about k8s, gets convinced they need and hires beforementioned devops to build it.

[goombaskoop]

> the average devops knows k8s and will slap it onto anything they see

This is certainly the case from all the third person accounts I hear. Online. I never actually met a single one that is like that, if anything, those same people are the ones that are first to tell me about their Hetzner setups.

[hkt]

DevOps here.

The trouble is that we are literally expected to do this everywhere we go. I've personally advocated for approaches which use say, a pair of dedicated servers, or VMs as in GPs example. If you want it outside of AWS/GCP/Azure, you're regarded as a crazy person. If you don't adopt "best practices" (as defined by vendors) then management are scared. Management very often trust the sales and marketing departments of big vendors more than their own staff. Many of us have given up fighting this, because what it comes down to is a massive asymmetry of information and trust.

[regularfry]

There is a kernel of validity lurking in the heart of all this, which is that immutable images you have the ability to throw away and refresh regularly are genuinely better than long-running VMs with an OS you've got to maintain, with the scope for vulnerabilities unrelated to the app you actually want to run. Management has absorbed this one good thing and slapped layer after layer of pointless rubbish on it, like a sort of inverse pearl. Being able to say "we've minimised our attack surface with a scratch image" (or alpine, or something from one of the secure image vendors) is a genuinely valuable thing. It's just the all of the everything that goes along with it...

[dijit]

Sure.

The challenge is convincing people that "golden images" and containers share a history, and that kubernetes didn't invent containers: they just solved load balancing and storage abstraction for stateless message architectures in a nice way.

If you're doing something highly stateful, or that requires a heavy deployment (game servers are typically 10's of GB and have rich dynamic configuration in my experience) then kubernetes starts to become round-peg-square-hole. But people buy into it because the surrounding tooling is just so nice; and like GP says: those cloud sales guys are really good at their jobs, and kubernetes is so difficult to run reliably yourself that it gets you hooked on cloud.

There's a literal army of highly charismatic, charming people who are economically incentivised to push this technology and it can be made to work so- the odds, as they say, are against you.

[vladvasiliu]

> If you want it outside of AWS/GCP/Azure, you're regarded as a crazy person. If you don't adopt "best practices" (as defined by vendors) then management are scared. Management very often trust the sales and marketing departments of big vendors more than their own staff. Many of us have given up fighting this, because what it comes down to is a massive asymmetry of information and trust.

I think this is the crux of the matter. Also, "everybody is doing it, so they must be right" is also a very common way of thinking amongst this population.

[nz]

The following happened to a friend.

Around the time of the pandemic, a company wanted to make some Javascript code do a kind of transformation over large number of web-pages (a billion or so, fetched as WARC files from the web archive). Their engineers suggested setting up SmartOS VMs and deploying Manta (which would have allowed the use of the Javascript code in a totally unmodified way -- map-reduce from the command-line, that scales with the number storage/processing nodes) which should have taken a few weeks at most.

After a bit of googling and meeting, the higher ups decided to use AWS Lambdas and Google Cloud Functions, because that's what everyone else was doing, and they figured that this was a sensible business move because the job-market must be full of people who know how to modify/maintain Lambda/GCF code.

Needless to say, Lambda/GCF were not built for this kind of workload, and they could not scale. In fact, the workload was so out-of-distribution, that the GCP folks moved the instances (if you can call them that) to a completely different data-center, because the workload was causing performance problems, for _other_ customers in the original data-center.

Once it became clear that this approach cannot scale to a billion or so web-pages, it was decided to -- no, not to deploy Manta or an equivalent -- but to build a custom "pipeline" from scratch, that would do this. This system was in development for 6 months or so, and never really worked correctly/reliably.

This is the kind of thing that happens when non-engineers can override or veto engineering decisions -- and the only reason they can do that, is because the non-engineers sign the paychecks (it does not matter how big the paycheck is, because market will find a way to extract all of it).

One of the fallacies of the tech-industry (I do not mean to paint with too broad a brush, there are obviously companies out there that know what they are doing) is that there are trade-offs to be made between business-decisions and engineering-decisions. I think this is more a kind of psychological distortion or a false-choice (forcing an engineering decision on the basis of what the job market will be like some day in the future -- during a pandemic no less -- is practically delusional). Also, if such trade-offs are true trade-offs, then maybe the company is not really an engineering company (which is fine, but that is kind of like a shoe-store having a few podiatrists on staff -- it is wasteful, but they can now walk around in white lab-coats, and pretend to be a healthcare institution instead of a shoe-store).

Personally, I believe that the tech industry sustains itself via technical debt, much like the real economy sustains itself on real debt. In some sense, everyone is trying to gaslight everyone else into incurring as much technical debt as possible, so that a way to service the debt can be sold. Most of the technical debt is not necessary, and if people were empowered to just not incur it, I suspect it would orient tech companies towards making things that actually push the state of the art forward.

[jcgrillo]

There was a moment ca. 2020 when everyone was losing their minds over Lambda and other cloud services like SQS and S3 because they're "so cheap!!11". Innumeracy is a hell of a drug.

[p_l]

Still is, just details change.

A lot of criticism of k8s is always centered about some imagined perfect PaaS, or related to being in very narrow goldilocks zone where the costs of "serverless" are easier to bear...

[WorldMaker]

> Personally, I believe that the tech industry sustains itself via technical debt, much like the real economy sustains itself on real debt. In some sense, everyone is trying to gaslight everyone else into incurring as much technical debt as possible, so that a way to service the debt can be sold.

This feels like a reminder that everything "Cloud" is still basically the same as IBM's ancient business model. We've always just been renting time on someone else's computers, and those someone else people are always trying to rent more time. The landlords shift, but the game stays the same.

[jimbokun]

“Nobody ever got fired for buying…

…IBM

…Microsoft

…AWS.”

[jcgrillo]

> Management very often trust the sales and marketing departments of big vendors more than their own staff.

They're getting kickbacks from cloud vendors. Prove me wrong.

[r_lee]

not sure if this is a thing with Cloud vendors, but e.g. in Finance, you'll definitely get the opportunity to call your rep over for free fancy dinners or whatever you want, because those are "customer meetings"

better than nothing, I don't blame em.

[jcgrillo]

> better than nothing, I don't blame em.

Money ain't got no owners, only spenders.

[ownagefool]

To be fair, I have k8s on my hetzner :p

[darkwater]

And the average developer doesn't even know where to start to deploy things in prod. When the feature product asks passes QA... to the next sprint! we are done!

[chrisweekly]

Whose responsibility is it to establish the prerequisite CICD pipelines, HITL workflows, and Observability infr in order for devs to shepherd changes to prod (and track their impact)? Hint: it's not the developer's.

[philipallstar]

This was the point of "devops" (the concept, not the job title): the team should be responsible for development and operations, so one isn't prioritised hugely over the other.

[liveoneggs]

But those things all require more pods on the cluster! We've looped back around to the beginning.

[darkwater]

Exactly my point. But then developers: "I just want to go to my Heroku days again!" but then with a sufficient big company there are maaany developers doing things their slightly different way, and then other effects start compounding, and then costs go up because 15 different teams are using 27 different solutions and and and...

But yeah, let's just spin-up a shadow IT VM with Debian like GP said, it's easy!

[throwup238]

> But yeah, let's just spin-up a shadow IT VM with Debian like GP said, it's easy!

That’s literally how they sold AWS in the beginning.

Cloud won not because of costs or flexibility but because it allowed teams to provision their own machines from their budget instead of going through all the red tape with their IT departments creating… a bunch of shadow IT VMs!

Everything old is new again, except it works on an accelerated ten year cycle in the IT industry.

[darkwater]

Indeed. And it stems from the illusion that what works in solo/small teams/scrappy startup works the same when you are bigger, and that a developer can take over all the corollary work to the actual product development.

And yes, a dev that's able to do that properly (stress on properly) is indeed a signal of a better overall developer but they are a minority and anyway as orgs scale up there is just too much of "side salad" that it becomes a separated dish.

[tete]

> the average devops knows k8s

If you'd know Kubernetes, you know not to use it. I say that as someone who used to do consulting for it.

The reality is that yet again "making money" completely collides with efficient, quality, sane productive work.

For me one of the main reasons to leave that space is that I couldn't really deal with the fact that my work collides with a client's success. That said I have helped to get off that stuff and other things that they thought they needed, that just wasted time and money. It just feels odd going into a company that hired you to consult on a topic only to end up telling them "The best approach for you is not doing that at all". Often never. Like some people thought "Well, if we have hundreds of thousands or even millions of users" and the reality was that even in these scenarios if you went away from that abstract thought and discussed a hypothetical based on their product they realized that they'd still be better off without it. Besides the fact that this hypothetical often was in a future that made it likely that they said they'd likely have completely different setup so preparing for that didn't even make sense.

I think a big thing related to that was/is the microservice craze where people end up moving to a complex architecture for not many good reasons and then they increase complexity way faster than what they actually deliver in terms of the product, because it somehow feels good. I know it does, I've been there. When in reality the outcome often is just a complex mess with what could have been a relatively simple monolith. And these monoliths do work. And in the vast majority of cases they are easy to scale, because your problem switches from "how do we best allocate that huge amount of very different services across our infrastructure" to (for the most part) "how do we spin up our monolith on one more server" which tends to be a way easier to tackle service.

And nothing stops you from still using everything else if you want. Just because it's a monolith doesn't mean you need to skip on any of the cloud offerings, etc. For some reason there seems to be that idea that if you write a monolith you are somehow barred from using modern tooling, infrastructure, services, etc. Not sure where that comes from.

[r_lee]

I think one big problem is that using microservice architecture doesn't mean that literally everything has to be a "microservice". if you don't truly need granual scaling (i.e. your "app" doesn't get a bunch of asymmetric loads across different paths), then you can just have more monolithic "microservices" until they need to be split up

imo this should achieve a nice balance?

[necovek]

Before microservices became trendy, there was something called SOA (service oriented architecture). Microservices is but one application of that pattern where each service is minimal.

SOAs have most utility in scaling teams, not software: creating independent services allows autonomy to independent teams if they apply a few simple patterns for good SOA.