[sheepdestroyer]

Comparatively, RHEL 6 is still kind of fine, at least it is still officially supported as virtualized OS in oVirt...

We run a lot of CentOS 5 virtual machines (and some physical ones! ; and some RHEL4! , and a few Fedora core 8 and 4 !!!), with no end in sight... :(

It is a huge concern for the Infra team, a source of many headaches, and we need to go through oops to keep them running, but:

- Clients don't want to move from OLDPRODUCT that requires extremely old php.

- Dev team is not interested in migrating OLDPRODUCT to a modern platform, or even try to put it in a container. Their eyes are turned to the shiny NEWPRODUCT that is seemingly never fully coming to production (only one client has signed for it).

- New clients are still regularly signed on OLDPRODUCT.

- No one in the org wants to pay for for a migration anyway.

- Since some clients have complained about poor apparent security, what was visible was just hidden behind newer haproxy.

[KronisLV]

> Dev team is not interested in migrating OLDPRODUCT to a modern platform, or even try to put it in a container.

Surely this wouldn't take more than 2 weeks: just figure out the install instructions for the old piece of software, rewrite them as a part of a Dockerfile (or similar set of instructions to build an OCI image, there are other options out there, too), setup some basic CI/CD which will execute Docker/Podman/... build command and update any relevant documentation.

I actually recently did that with some legacy software, apart from additional optimizations like clearing yum/dnf cache after installs, it was pretty quick and also easy to do! If you are also in a situation where you need to use EOL software, I don't think there are many other decent options out there, short of trying to reproduce the old runtime on a new OS (as others suggested).

Running the old EOL OS will simply leave you with more vulnerabilities with no fixes in the long term and is generally a really bad idea. How did your security team greenlight that idea? In some orgs out there the devs would be made to stop doing whatever they're doing (outside of critical prod fixes) and would be forced to migrate to something newer before proceeding.

[capableweb]

> Surely this wouldn't take more than 2 weeks

Which, based on what previous commentator said, is just about 2 more weeks than anyone want to spend on it.

If no one wants to do it, it doesn't matter if it takes 2 days or 2 months, it won't get done.

[bombcar]

And if you take it upon yourself to try, suddenly you become the sole point of contact for anything that ever goes wrong with it from then on, even if it would have failed before the change.

[semi-extrinsic]

OT, but: why don't you just compile up an old php version from source on a new OS? It's a bit of a hassle the first time you do it, sure, but less than the hassle of running multiple legacy OS?

[bombcar]

The legacy OS is even less hassle, because it never gets any updates and just sits there.

If you compile old software on a new OS, every single update to that OS has a chance to blow up your compiled old version, so it takes more hand-holding.

Can be worth it at times, but other times it's just easier to firewall and hope for the best.

[submeta]

This sounds so familiar. Running an old OS that gets no updates and patches because of an app that relies on an old php version, also the fear of migration costs and an open ended project that never finishes. This is so wide spread it becomes a meme.

[perryh2]

Are you sure you want to disclose your employer's security risks and tech debts on a public forum?

[KronisLV]

I think that as long as the person remains mostly anonymous (personally I've no interest in digging further, for this exact reason), things like this need to be talked about.

Sure, lying about some of the details while getting the gist across might be a good idea (e.g. naming a different OS when you actually have Ubuntu), but there definitely needs to be discourse about the circumstances that people are dealing with in reality, instead of everyone pretending that they are on the forefront of the industry with their security and other practices.

The more you think about it, the murkier everything gets - talking about how things were in a company 5 years ago that's now defunct might be the best possible circumstance, but in a currently active org that might also be a way of getting yourself sacked, depending on how juicy of a target it is and how much attention it attracts.

That said, the company has made the choice to use EOL software, their clients have made the choice to use EOL software and it's bad practices all around - honestly, should any of the software be exposed publicly, I'm pretty sure that this person's comments won't be the first to call attention to the setup, since nation X's hackers/crackers might have already ran automated attacks against it anyways.

In summary, bad practices probably deserve to get called out, just so we know what the situation in the industry is like in reality, but hopefully not at too great of a personal expense.

[2b3a51]

https://lwn.net/Articles/863008/

About 1 million CentOS 6 boxes of some description hitting the update servers as of July 2021. Compared with roughly 2 million on v7 and half a million on v8.

Would be interesting to know how those percentages have changed since then.

[bombcar]

This is one of the reasons why if we ever touch a box (say to update whatever it is doing) we also bring it to the latest possible version of the OS we can find; luckily for us CentOS exploded just before the last one, so we moved to Ubuntu instead of CentOS 8 which was our original plan.

[jacquesm]

They could well be talking about any company that has multiple legacy products from two decades ago still running, which is quite a few of them...

[ChuckNorris89]

Yeah, let's not underestimate how common this is. Narrowing it down would be neatly impossible. And that's even considering parent was fully truthful which he might not have been to preserve anonymity.

[speed_spread]

I think they know exactly what they're doing. Nothing bumps an upgrade up the backlog like an attack on an old cashcow.

[lima]

They didn't say who they were working for, so why not?

[carlmr]

>NEWPRODUCT that is seemingly never fully coming to production

>- New clients are still regularly signed on OLDPRODUCT.

I mean what's the WHY behind that? Why don't even new customers sign on to the new product? Why is the new product not in production? Is that the same reason?

[bombcar]

NEWPRODUCT is almost always designed by Sales and OLDPRODUCT is old enough to have enough actual users that it works and does what is needed.

The end result is usually to force everyone to move to NEWPRODUCT and deprecate OLDPRODUCT violently (Salesforce Lightning vs Classic, etc, etc). Hopefully enough fixes for NEWPRODUCT get done before all the customers leave.

[LinuxBender]

I am too familiar with this dilemma. A work-around that comes with some caveats is to disable loading of modules. [1] There are more caveats than the article mentions including rebuilding initrd/initramfs can break unless you first reboot. Do not set the settings they mention in a persistent file like sysctl.conf.

[1] - https://linux-audit.com/increase-kernel-integrity-with-disab...

[mschuster91]

> - Clients don't want to move from OLDPRODUCT that requires extremely old php

I know that problem for a thankfully long gone Java internal application, and well... I went with running old stuff in `debian/eol` Docker containers [1]. Turns out you actually can use Docker as a sort of extremely lightweight VM service.

[1] https://hub.docker.com/r/debian/eol/

[josephcsible]

> at least it is still officially supported as virtualized OS in oVirt...

If they're not providing security fixes for RHEL 6's packages, then why does this matter?