[andrewguenther]

In 2025 there's no reason for anyone to be logging into an AWS account via the root credentials and this should have been addressed in the preventative measures.

There's no actual control improvements here, just "we'll follow our procedures better next time" which imo is effectively doing nothing.

Also this is really lacking in detail about how it was determined that no PII was accessed. What audit logs were checked? Where was this data stored?

Overall this is a super disappointing postmortem...

[nerdjon]

> In 2025 there's no reason for anyone to be logging into an AWS account via the root credentials and this should have been addressed in the preventative measures.

I am curious what preventative measures you expect in this situation? To my knowledge it is not actually possible to disable the root account. They also had it restricted to only 3 people with MFA which also seems pretty reasonable.

It is not unheard of that there could be a situation where your ability to login through normal means (like lets say it relies on Okta and Okta goes down) and you need to get into the account, root may be your only option in a disaster situation. Given this was specifically for oncall someone having that makes sense.

Not saying there were not failures because there clearly are, but there have been times I have had to use root when I had no other option to get into an account.

[oneplane]

You don't need the root account, unless you need to bypass all policies. In such a scenario, you a use the root access reset flow instead, reducing standing access.

As for other flows (break glass, non-SSO etc), that can all be handled using IAM users. You'd normally use SAML to assume a role, but when SSO is down you'd use your fallback IAM user and then assume the role you need.

As for how you disable the root account: solo accounts can't, but you can still prevent use/mis-use by setting a random long password and not writing it down anywhere. In an Org, the org can disable root on member accounts.

[nerdjon]

To me that sounds like security by obscurity not actual security.

If you have the ability to go through the reset flow than then why is that much different than the username and password being available to a limited sets of users. That would not have prevented this from happening if the determination was made that all 3 of these users need the ability to possibly get into root.

As far as having an IAM user, I fail to see how that is actually that much better. You still have a user sitting there with long running credentials that need to be saved somewhere that is outside of how you normally access AWS. Meaning it is also something that could be easily missed if someone left.

Sure yes you could argue that the root user and that IAM user would have drastically different permissions, but the core problem would still exist.

But then you are adding another account(s) on top of the root account that must exist that you now need to worry about.

Regardless of the option you take, the root of the problem they had was 2fold. Not only did they not have alerts on the usage of the root account (which they would still need if they switched to having long running IAM users instead, but now they would also need to monitor root since that reset flow exists) and their offboarding workflow did not properly rotate that password, which a similar problem would also exist with a long running IAM user to delete that account.

At the end of the day there is not a perfect solution to this problem, but I think just saying that you would never use root is ignoring several other issues that don't go away just by not using root.

[oneplane]

Not using root means not bypassing policies. There is no way to not bypass all policies. So yes, never using root makes that issue go away completely.

As for all the other stuff: what it does is it creates distinct identities with distinct credentials and distinct policies. It means that there is no multi-party rotation requires, you can nuke the identity and credentials of a specific person and be done with it. So again, a real solution to a real problem.

[nerdjon]

It solved “a” problem buy not “the” problem.

It depends on what the goal of all of this was, which is unclear. If the goal was simply to get the data that they originally wanted it does not solve that problem and it would have just happened a different way.

According to the article there was 11 days between the first actions taken and them finding out it happened.

If instead of a root account you have a long running IAM user that you can then assume into the role you normally use through SSO. If you also do not monitor that account with proper alerts and proper offboarding procedures than they could have logged into that account and retrieved the data they wanted.

Which again is the reason I am saying they just saying not using root is not a magic bullet that would have avoided problems. Maybe the situation would have been different but they still could have done a lot in 11 days.

[oneplane]

The problem was that the user's credentials were revoked but because the root account was a shared credential it wasn't revoked. Was the break-glass account also a user-specific account, it would have fit in with any 'revoke anything for user XYZ' workflow instead of being a root account edge-case.

So, in short, this would likely have prevented this, as the normal off boarding for user-bound credentials worked out fine already.

[zrail]

Resetting the root password requires proving access to the email address associated with the root account. It also leaves a massive papertrail.

[akerl_]

By "massive papertrail" do you mean "a pair of emails to the associated address"?

[oneplane]

No, a massive amount of CloudTrail logs.

[flumpcakes]

I'm now questioning my sanity but I thought you could disable login for the root account in AWS.

[the_mitsuhiko]

Since there are certain operations that can only be done with the root account, there is no way to disable access to it.

[jhealy]

Since 2024 you can disable the root credentials on all accounts except the Organization management account: https://aws.amazon.com/blogs/aws/centrally-managing-root-acc...

I don't think the post mortem details whether the root access was on the org management account or an org member account.

[the_mitsuhiko]

Oh wow. I completely missed that change.

[brian_cunnie]

Sometimes I log into the root account to see the billing information.

I created an "administrator" account, but apparently it can't see the billing information, including the very-important amount of remaining cloud credits.

Maybe I could spend time fiddling with IAM to get the right privileges, but I have more pressing tasks. And besides, on my personal AWS account I only log in with the root account.