| > The old prototype machine had our AWS API access key and secret key. Once the hacker gained access to the keys, he created an IAM user, and generated a key-pair. Ouch! This is why you must practice the principle of least privilege when provisioning access keys, especially ones that can control your entire infrastructure. > He was then able to run an instance inside our AWS account using these credentials, and mount one of our backup disks. Good security practices are like an onion ... it's got many layers and will most likely make you cry. This is why you should be encrypting your backups. The backup program doesn't even need to be able to read the backups it creates, it only needs to to be able to write them (i.e. the public half of the key). If you're thinking, "Why does that matter? It can already read the plain-text data it's backing up!" then you're not thinking about history or multiple servers. Each may be able to read the current state of its own file system, but if they only have the public half, they can't read the historical state or each other's backups. The damage would be limited to a single server and only what's live on it at that moment. > We were able to verify the actions of the hacker using AWS CloudTrail, which confirmed that no other services were compromised, no other machines were booted, and our AMIs and other data stores were not copied. CloudTrail is awesome and if you're on AWS you really should enable it. It costs peanuts compared to what it provides. In cases like this, where the attacker takes over your entire infrastructure, it may be the only log you have of what happened. |