Hacker News new | ask | show | jobs
by capableweb 1535 days ago
CircleCI is a CI/CD solution (ala SaaS) that you don't host yourself. Many (myself included) prefer to host mission-critical services ourselves to avoid untimely downtime.
3 comments

Hosting any significantly scaled CI/CD system is very difficult work and since every team in your organisation usually touches it you often have a large blast radius. I've seen downtimes take a long time to recover from even when self hosted, compliance is also hard. It's a good candidate for SaaS IMO.
I've been the owner of our ~90 user GitLab deploy and its been mostly painless over the past two years, we have it installed on an autoscaling GKE cluster subscribed to the 'stable' channel. I helm upgrade it monthly.
You have a strange way of saying "self-hosted" ;) But it sounds like you've found a happy medium. This is a subject I may need to look into the in relatively near future.
As mentioned in a sibling comment: Yes, self-hosting comes with it's own problems, there are no silver-bullets in this industry.

However, I'd still argue that choosing when downtime can happen is important when you're pushing out larger changes to larger organizations. You don't want to be in the middle of a borked migration when your CI/CD service craps out and you can't rollback/push more updates or even restore backups as it was all automated via your CI/CD service.

I know what CircleCI is. Just wanted to understand what part of it you were referring to, which seems to be SaaS in general. I'm honestly not really convinced about self-hosting really avoiding untimely downtime, but whatever works for you and your team.

E.g. I've worked in a business who self-host their Gitlab instance, there was a non-negligible amount of work for backups/upgrades on top of troubleshooting performance issues once in a while, amongst other things in the same vein.

The part about "untimely downtime" is about that CircleCI decides themselves when to push updates (which is the most common reason services has downtime), instead of you deciding when to upgrade/push updates. If you have a big migration/change coming up, you'd put pause on upgrading the CI/CD service as you don't want to muck with it while pushing out other organization-wide changes.

Granted, self-hosting comes with it's own share of problems too, no solution is a silver-bullet without any issues, but being able to "freeze" things to a stable mode helps to stabilize other processes.

Every service will have down time. It is about the locus of control and agency. When I self host I can decide when to apply updates and do maintenance. When something does go wrong I can dive in, fix the problem, and move on with my day.

Most people don't want any responsibility though so they do everything they can to push work off to a SaaS or cloud provider. That way when the SHTF they can browse reddit and point their boss to the status page.

> Most people don't want any responsibility though so they do everything they can to push work off to a SaaS or cloud provider. That way when the SHTF they can browse reddit and point their boss to the status page.

Phew, those are a lot of assumptions. Many businesses choose cloud services cause they take the tradeoff between that cost or having someone at hand to maintain the infrastructure, depending on its complexity. I've worked with employers using many cloud providers for years now, the amount of times I would _legitimately_ be prevented from working at all by outages is extremely low.

The agency part, I do get. But come on.

They offer to self-host by the way.