Hacker News new | ask | show | jobs
by mtlynch 1249 days ago
This is a good writeup, and I appreciate the transparency. I especially like this bit:

>While one employee’s laptop was exploited through this sophisticated attack, a security incident is a systems failure. Our responsibility as an organization is to build layers of safeguards that protect against all attack vectors.

I was surprised by this part:

>To date, we have learned that an unauthorized third party leveraged malware deployed to a CircleCI engineer’s laptop in order to steal a valid, 2FA-backed SSO session.. the malware was able to execute session cookie theft, enabling them to impersonate the targeted employee in a remote location and then escalate access to a subset of our production systems.

I'm surprised the SSO session token isn't bound to an IP address. I'd also expect access to prod overall to be whitelisted to CircleCI-owned IP ranges.

Now some gripes:

* I never received an advisory email about this incident. I only received this follow-up to one of my Github machine accounts, not my primary billing account.

* Their secret-finding script is pretty bad. It just dumps out a bunch of metadata without helping to make it actionable. Environment variables still don't have a created_at field, so you can't verify which ones you might have missed in a broad key rotation.

2 comments

Is tying session tokens to IPs actually common? I can't imagine it is given the unreliability of IP addresses causing issues.

I used to live somewhere where outbound traffic went through one of three CGNAT IPs at random, and I only had auth issues with one really old site that predates the NAT hell that is the modern internet.

Yeah, good point. I guess it'd be a pain to have to keep reauth'ing if your IP changed for legitimate reasons.

It would be possible to do some kind of check for "this session token was used in the US and Russia twenty minutes apart... something's fishy," but that adds in more complexity.

session vs refresh… you kill the session token and require a refresh. Can be sometimes be transparent, but may cause a re-authentication using the second factor with an indicator to the user that their previous session was killed due to use by a different IP.

If you are concerned about stable IPs, use a proper VPN or bastion setup.

CircleCI is 100% remote. I can't imagine having to keep up with everyone's constantly changing home IPs and making exceptions while travelling.
I'd imagine they have a VPN.
But they must not be requiring it to access these admin systems. Not mentioned in the write up either, surprisingly
I’d imagine they do now… at least for privileged account usage.