[setquk]

Jenkins is one of those things that looks good until you trip over the numerous bugs in it. Such as the slightly terribly leaky implementation that hoses itself once a week, the severely million inodes it decides to gobble up which makes backups a chore etc.

Sure it works but from an admin perspective it’s horrid.

[peterwwillis]

Is this really more horrid than having to hire a DBA to maintain, operate and upgrade the cron replacement? I can write a script to identify and remediate bugs in apps. I can't so easily write a script to diagnose badly used database apps and get the developers to rewrite their queries to not blow up the database servers. It's much easier to set up, lock down, and maintain Jenkins than Postgres+whatever, imo. But I guess everything can go to hell depending on how it's used.

[coldtea]

>Is this really more horrid than having to hire a DBA to maintain, operate and upgrade the cron replacement?

Only you don't. I'm not sure why you seem afraid of Postgres, but millions of people use it, across tons of companies, and even for personal projects, and it's trivial to setup and run. Oh, and those lots of plugins and stuff you mention? You don't have to use them if you don't need them. They don't even enter the picture at all.

[peterwwillis]

> I'm not sure why you seem afraid of Postgres

Because I've managed database applications before?

> and it's trivial to setup and run

Jenkins is more trivial. It's one process. It doesn't depend on an external high-availability networked data service. Backup is 'cp -r SRC DEST', or a plugin if you're fancy. And, again, Postgres does not replace Jenkins, it's just the storage and querying. It adds a lot of complexity and service availability points of failure that Jenkins does not have.

[user5994461]

Last I checked. The plugin to do backup didn't work.

Copying the jenkins directory can be done but restoring it on another fresh install will not work. There are many files that needs to be deleted manually until the instance can start without error.

[setquk]

We have over 100Gb over tens of millions of inodes in Jenkins. This does not scale well even on XFS backed with SSDs. Takes hours to run backups.

pg_dump on the same data volume takes about 14 minutes to dump, compress and move to another node.

The filesystem is a shitty database. Thought we’d all learned that by now.

[mmt]

> The filesystem is a shitty database. Thought we’d all learned that by now.

How dare you :) "The" filesystem is a great database.. for certain applications. Big, binary blobs of video, store, and even index, particularly well!

I think what the collective "we" haven't learned is to avoid trying to think about scale intuitively (rather than "doing the math") and to avoid extrapolating from the trivial scenario.

It's why "latency numbers every programmer should know" is still a thing.

[setquk]

It’s ok for the narrow set of use cases that it is good at. Outside of that it needs something that has some intelligence and ability to read optimise it.

Heavily locked stuff, lots of small things, huge number of locatable data entries, not so much. Which is Jenkins.

As for scale, Incidentally I worked on a very old filesystem based store back when we had spindles to contend with. We had four racks of Sun disk array trays. The only way it performed well was keeping it to 2Gb spindles. I’m well aware of scale issues on file systems, perhaps moreso than the Jenkins developers. We had to scale that to 4000 concurrent users.

[setquk]

I’m a postgres DBA as well. It mostly requires zero maintenance.