Hacker News new | ask | show | jobs
by oneshoe 2814 days ago
How to successfully roll out a product/site.

Released "Version 2" which was an entire newly built system that we worked on for an entire year (we also thought we were "agile" btw). This completely crashed and around 10K users were unable to use the product for almost 2 weeks. We did plenty of things right but we also failed.

Biggest Lessons learned: 1. Release features and changes in manageable incremental chunks 2. Load test, load test, load test - just do it! Yeah, sure, your app worked with your one user when there is 1kb of data in your datastore - what about 5000 users accessing it with 10GB in your data store? 3. Get and use an APM - when it feels like everything is on fire you'll need to know where the fire is coming from 4. One way changes that can't be rolled back are the devil. Take the time needed to make sure you can roll back - manageable incremental changes help with that.

2 comments

My project management muscles arr being trained to be risk averse, and my 'move fast and ship ' muscles are kicking in to make releases multipart and piece meal.

It werks

What is an APM in this context? Google just tells me about Alternative Payment Methods.
Application Performance Management/Monitoring.

New Relic, AppDynamics, and DynaTrace are a few vendors in this area.