[kogepathic]

> I would never approve a production system as expansive as Gitlab's to only have two databases in a cluster. That is asking for trouble, and any {sys,db}admin worth their salt will tell you the same. As soon as you need to do anything on one database, you've just lost your cluster policy.

I do agree with you it reflects poorly on GitLab for only having a primary and replica, with broken backups.

BUT, they are a startup, and they need to be laser focused on growing the company and securing funding for the next quarter so they can keep the lights on.

This kind of pants on fire growth in a startup often (always?) comes at the cost of redundancy and best practices. If you stop to make your platform bulletproof instead of the new features you promised customers/investors, you die.

I'm not saying this is an excuse for them to permanently shrug off making their platform more redundant and engaging in best practices. But, as with many startups, they're focused on delivering features as fast as possible to grow their user base.

I think, and hope, that their recent outage has been the experience they needed to prioritize their shift toward more redundancy and best practices.

[lightedman]

"BUT, they are a startup, and they need to be laser focused on growing the company and securing funding for the next quarter so they can keep the lights on."

Having run and sold a few startups and turnkey operations, let me tell you - if you don't focus on your core product first and foremost and demonstrate the utmost competency in it, you're just pissing money away and are likely to fail.

[kogepathic]

> if you don't focus on your core product first and foremost and demonstrate the utmost competency in it, you're just pissing money away and are likely to fail.

I don't have direct experience running/selling a startup, but I have worked for several as an employee.

In my experience, the product is important, but not necessarily the deciding factor in whether the company succeeds. Most of the success is down to the management team (e.g. CEO, CTO, CFO) being able to sell the company (and product) to investors during funding rounds.

Doesn't matter how good the product is, if management can't pitch it to get funding you're dead.

Obviously it is important to have a good product, but just look at Uber for example. Losing billions of dollars per year, with no clear path to profitability, and yet they're valued at something like $60B. Great sales/marketing by their management to investors!

[lightedman]

"Doesn't matter how good the product is, if management can't pitch it to get funding you're dead."

Good products literally fund themselves and don't need outside investors. Just like drugs sell themselves, a good product will sell itself without the need for anything like marketing or investors. The money will naturally come.

[sytse]

For sure it caused us to prioritize more redundancy, for more information please see https://about.gitlab.com/2017/02/10/postmortem-of-database-o...