Hacker News new | ask | show | jobs
by lmilcin 1649 days ago
On my infrastructure the number of people with direct access to modify the database is zero.

What you do is you write any operations that are to be run on production as code (for example service that can be triggered).

You then get the code reviewed (second pair of eyes) after which it is automatically compiled and delivered to pre-production. On pre-produuction you have a chance to test it against full environment to see if it is breaking anything. If tests results are accepted it can be promoted to prod.

We actually have special service that is only meant to run one-off database operations. It is a webservice where you can trigger jobs.

Jobs do not take any parameters at all -- you just give the name of the job and that's it. This allows the entire operation to be reviewed in full.

Jobs can't run in parallel -- there is a queue and you can schedule a job to run. The job can also be saving progress (and it will restore from the saved progress in case it fails or is cancelled mid-operation).