Hacker News new | ask | show | jobs
by silverfox17 1372 days ago
This seems a bit generalized. Yes, infrastructure as code/etc. are becoming more prevalent, but underneath there are still systems running.

While a lot of jobs can abstract away these things, they are still there, and very real. If more people had an understanding of the underlying operating systems and file systems we could mitigate a lot of vulnerabilities, or find performance issues that may otherwise be obstructed, or a myriad of other things.

Hacker News is in a bit of a bubble in that the answer to everything seems to be "kubernetes" or "$newHotSolution". I think part of it is that a lot of developers haven't actually worked with machines. At one point in time there was a post here about how hard it is to set up a LAMP stack, a task you could hand a first year sysadmin and they should be able to figure it out. Abstraction and automation are nice, but the underlying concepts are still important.

3 comments

Comparing kubernetes to a sysadmin manually provisioning a LAMP stack is like comparing a home kitchen to commercial factory. They can both make a pizza, but one can make an order of magnitude more - at same time.

They’re solving two different problems.

They are, but a lot of people think they need a commercial kitchen to make a bagel now a days, because they've only worked in commercial kitchens.
We’re in agreement there. Right tool for the job.
Developers deploying static website in 2022 using K8S be like: https://xkcd.com/1319/
I'm a dude who uses Kubernetes to make pizzas. Kubeternetes is absolutely a commercial bakery class machine, but much of it's adoption is due to the fact that for just a bit more in price and effort, you can have that class of machine in your home and run real things on it.

Seriously: I run clusters from a few dozen nodes (down from a few hundred at my peak, sigh) down to a trio of Raspberry Pis in my living room. They're overkill by a little bit, but not by much. And it's definitely my ambition to make the tooling even easier and even more powerful, such that every small home can run something with enterprise level reliability.

I'd say Kubernetes was more like having a lathe in your shed. Almost no-one needs one but it sure does make some projects a lot easier.
I just got one and not sure I need it. The lathe always seemed so dangerous to me.
Of topic but having a lathe in my shed is a definite life goal.
Wrong comparison, sorry. K8s is enterprise thing, while LAMP is good for SOHO. So cookies factory vs small bakery.
This is a poor analogy overall, but I think to would be better to think of it thusly...

K8 is where you don't own the kitchen, and you lease it when you wish to cook. You aren't aware of how to maintain the kitchen, or buy the ingredients you choose to cook with. In fact, you aren't even able to tell if a mango is ripe or not when shopping, because you don't shop and don't know how to.

That's what K8s are.

Meanwhile, SysAdmins know how to maintain, manage, run the kitchen... as well as cooking the meal. SysAdmin knowledge scope is greater than K8 knowledge scope for this reason.

What AWS, what docker, what K8s have done, is outsource specific realms of knowledge and skills, so people don't have to "deal with that". But if one is outsourcing knowledge and skills, one cannot claim that this makes the work more sophisticated.

A couple of examples I've found:

Celery spawns n workers, defaulting to the number of logical processors available. As anyone familiar with cgroups can tell you, this is fraught with problems when containerized, since nearly all mechanisms to detect processor count (or available memory) lead back to `/proc`, which will dutifully report the host's information out to the container. This leads to questions like, "I requested 4 vCPUs; why do I have 4+n threads?"

ORM in general. The worst example I've seen was querying a massive table, and to get n results from the end, was using OFFSET + LIMIT. It was also deliberately not using the PKEY, leading to a full table scan every time the query ran. If you aren't familiar with DBs, it may seem perfectly reasonable that querying a ~100 million row DB would take a long time, when in fact it could and should be extremely fast with a properly written query.

You do realize some of the internet's biggest sites run on the lamp stack, right?
With Apache httpd inside? They don’t have devops at all?
not really. its a spectrum.

the only real difference is the sense of smugness when it works. In the old days deploying LAMP was a sense of achievement. save for patching, there wasn't much more work to do.

Kubernetes is basically the same level of effort, but the upkeep is a bit more.

Also the networking is batshit, and so is the aversion to swap.

Doesn't seem to me that there's a bubble here where the answer is always Kubernetes because everytime that topic pops up there are a lot of posts like yours.
Another part of the problem is that developers are often discouraged or outright not allowed to work with machines, due to "it's not your job" kind of arguments, or corporate security enforced by auditors.
You don't need corporate security or auditors; even in a small web-shop, administering production servers is a power reserved for wizards. Sure, you can administer your own developer workstation; perhaps you get a Linux VM to yourself, that you can tinker with.

It would be nuts to let every developer tinker with the production server, or the source-code repo, or the fileserver. The private VM gives them a playground where they can learn to play at sysadmin, if they want; real sysadmin, I would contend, is taking the burden of responsibility when shit happens. Nobody cares much about wizards, until shit happens; the wizard then becomes an essential scapegoat. The newsgroup alt.sysadmin.recovery wasn't created for nothing.

> It would be nuts to let every developer tinker with the production server, or the source-code repo, or the fileserver.

I think that it might be a good idea to have most of the configuration for servers be based on Git repos, with something like Ansible or another such solution (Chef, Puppet, Salt), so that server configuration wouldn't be treated that differently from versioned code (which also serves as a historical record).

Don't give developers access to just push changes for production servers, but absolutely let them create merge/pull requests with changes to be applied once proper review is done: ideally with a description of what these changes accomplish, references to other merge/pull requests for development/test/staging environments where they were tested beforehand (perhaps after prior tests against local VMs) and a link back to whatever issue management system is used.

Then, have an escape hatch that can be used by whoever is actually responsible for keeping the servers up and running, in cases an update goes bad or other manual changes are absolutely necessary, but other than that generally disallow anyone to access the server directly for whatever reason (or at least remove any kinds of write permissions).

Personally, I'd also argue that containers should be used to have a clear separation between infrastructure and business applications, but that's mostly the approach to use when dealing with web development or similar domains. Regardless, I find that it's one of the more sane approaches nowadays, the Ops people can primarily worry about the servers, updates, security, whereas the developers can worry about the applications and updates/security for those.