Hacker News new | ask | show | jobs
by ShakataGaNai 806 days ago
That's great that DevOps (or whatever their title) owns both product and marketing sites. Far too many companies (and DevOps teams) think the www site is "not important" or "not their core job" and outsource it to either a less qualified team, or out of the company altogether.

From an external perspective no one cares if www going down isn't "your fault" or of "direct impact to the product". It's a corporate blackeye either way.

3 comments

This hasn't been my experience. In my experience, the reason why your ops teams divest themselves from the marketing side is because marketing decides to contract some firm to design their site for them, and the firm decides to deploy to Vercel or WP-Engine or whatever. Then, Marketing comes to ops and says "hey, I need you to change this DNS thing" weeks/months into their engagement, with no understanding of the ramifications. Ops/product team pushes back, because the change would fundamentally break the application. Marketing gets defensive, "we've spent all this time and money on this, you just need to make it work", a broken halfway solution is implemented, and ops/product, in protest, divests themselves from the solution. Bingo bango, shadow IT is ratified, the kludgy hackjob lives in production forever, and no one thinks about it until the next time something breaks.

Reminds me of the time marketing decided to change the logo on the marketing site for the product team I was on without being aware that the site was scraped and redeployed on a different domain (by hand). When the logo changed, the CSS for the image element wasn't updated, truncating part of the logo, proudly displaying the word "ass" as a part of the logo in an unfortunate cropping incident.

"Far too many companies (and DevOps teams) think the www site is "not important" or "not their core job" and outsource it to either a less qualified team, or out of the company altogether"

It's impossible to know because they won't admit it publicly. You are guessing based on some anecdotal experience.

But then again... here's mine! I worked at a very successful SaaS that had (really not kidding) the most incompetent, lazy dope running the www site. He live-edited a "staging" version of the site on the fly (no, it wasn't private, you could access this thing from the internet, and he didn't know or care about that). When he was happy with his changes he'd destroy the live instances behind the load balancer and clone his staging instance without taking it down or running any extra checks. This staging instance was around for years and I don't think he ever bothered doing a system update. Since he didn't use git, I I'll bet that at least once he cloned a live instance back to staging to undo a bunch of bork.

I lost count of the incidents. He never detected them himself, was never available to troubleshoot them and was generally a big "durrrr" when you'd finally get him on the call. Example: one time we had a "slow, intermittent errors" customer support ticket surfaced to us, not because it was our job, but because dopey was being an absolute ass to the helpdesk guys. He ran his crap in another AWS account we didn't have access to. About a day later the www site went down completely, so we got hold of the AWS account and dug in. All 5 of the instances behind the load balance were "unhealthy" for various reasons. Certs expired, disks full, apache stopped. We bounced them, restarted them and sshed in. They all had different versions of the site. It was a complete mess. Turns out dopey wasn't very good at killing the old instances and cloning staging. He was probably live-editing the instances for smaller changes if that seemed easier than a bunch of AWS console work.

Unbelievably he wasn't fired and continued to mismanage the site, and we could do nothing because the head of marketing didn't listen to the head of engineering. They hated each other. The way Marketing saw it "your SRE guys couldn't fix it, they had to wait for <dopey> to get on the call". I'm not even kidding.

Just more anecdotal evidence from me. You might be right.

Alas, the part you're describing as great is written in past tense.