| I've seen mentioned in the comments various "deployment" tools (or call them "configuration management" if you will) being called "insecure" or "immature", or one being claimed better than another; however I think this is a good opportunity to talk about a deeper problem, namely the architectural choices each tool has taken. These choices all impact the reliability and security of the resulting system, especially the following: * do they rely on SSH, or they have implemented their own authentication / authorization techniques? (personally I would be very reluctant to trust anything that just listens on a network port for deployment commands, and it's not SSH;) * do the agents run with full `root` privileges, or is there a builtin mechanism that allows the agent to act only in a limited capacity, within the confines of a set of whitelisted actions? (perhaps even requiring a secondary authentication mechanism for certain "sensitive" actions, for example something integrated with `sudo`, that provides a sort of 2-factor-authentication with a human in the loop;) * do the operators have enough "visibility" into what is happening during the deployments? (more specifically, are the deployment scripts easily auditable or are they a spaghetti of dependencies? are the concrete actions to be taken clearly described, or are they hidden in the source code of the tool?) * are there builtin mechanisms to "verify" the results of the deployments? * and building upon the previous item, are there mechanisms to continuously "verify" if the deployment hasn't changed behind the scenes? I understand that some of these features wouldn't have helped directly to prevent this particular case, however it would have helped in alerting and diagnosis. |