[hn_throwaway_99]

I really appreciate detailed breach reports like this. This was the money quote for me:

> The attackers continued their search and eventually discovered a mounted NFS share on the web server. This file share contained notes and configuration files used by Equifax engineers, in which they found many database credentials.

Seriously, WTF? I get paranoid all the time worrying about my application security - it often feels like there is always some potential issue around the corner you don't know about.

But then I read about how lots of these kinds of breaches occur (storing prod DB credentials in plaintext on an NFS share, reusing passwords and not using 2FA, leaving your server password as "solarwinds123", etc.) and I think maybe I'm not so bad after all.

[keyle]

I'm totally with you on this one; but remember, if you have 25 developers in groups of 5, in only takes 1 muppet in any of the 5 groups to have low standards, and voila.

I've seen it, in pretty much every large business I've worked in.

This goes back to the saying: "you should never hire someone less good than yourself".

Sadly when the people hiring literally come from sales or airline customer service, your company is boned. It's only a matter of time.

[hn_throwaway_99]

I agree with all that, with the one small caveat that more than anything else I think what is most important about security is a strong security culture at a company. All the checklists and compliance frameworks in the world are doomed in the face of a poor security culture. On the flip side, a strong and constantly reinforced security culture can help protect against the occasional muppet.

One example: years ago I started work at a tech company (a fintech no less), and shortly after starting I asked the head of customer service how I could get an account to access an internal admin portal (I was an engineer and needed to understand some of ops processes). "Oh, you just log in with my account, and the password is <CompanyName><Year> - all the reps just use that shared account" I got an immediate sinking, sinking feeling of despair.

[foota]

Better than culture is enforced guarantees, nobody can store the database password on an NFS share if it's not available to them.

[hn_throwaway_99]

I honestly believe that enforced guarantees come about through security culture though.

Meaning a strong security culture means you do appropriate secrets management, and importantly, everyone understands how secrets management should be done. That way if you have the occasional breach in your automated enforced guarantees (e.g. the article talks about how Equifax missed one of their vulnerable systems to patch), that if people see a problem they will speak up.

That is, I agree with enforcing guarantees as much as possible, but any engineer on that team who came across an NFS file with DB credentials should have spoken up loudly about "Why TF are these DB credentials present on a network drive?"

[mcny]

I think you will love the way Microsoft handles this. Basically, there are automated flags that seem to ding managers (M1 I believe) so they will make sure the people in their teams handle these.

> any engineer on that team who came across an NFS file with DB credentials should have spoken up loudly about "Why TF are these DB credentials present on a network drive?"

This requires empowering your employees and the lower case a while with its cross functional teams which most managers hate.

[eli]

If you take away NFS shares without providing a better way to store and manage access controls, engineers will eventually just come up with an even worse solution.

I'm skeptical you can even fix this without a culture change, but you definitely can't do it just by taking things away.

[foota]

Yes, I agree. I realize this isn't feasible everywhere, but having access tied to a user account (and then auditing and limiting that access) can serve as a replacement. E.g., want to select a single row? Fine, but if they're dumping the db something is phishey.

Ironically, user accounts are in one sense more secure (than a system account with a shared password) because they can use 2fa (and there's no inherent need to distribute the password).

[mysterydip]

All it takes is outsourcing one portion to the lowest bidder, and you get what you pay for.

[rawgabbit]

What is egregious is that Equifax isn't some random company. Equifax is a credit reporting agency and is a goldmine for identity thieves.

After the Equifax breach, I just assume now if an identity thief gives a half assed effort, he can pull up the PII for any American resident. How Experian/Equifax/Transunion can honestly say they have accurate data without physically verifying driver licenses, identity cards, or passports is beyond me.

[coldpie]

> This goes back to the saying: "you should never hire someone less good than yourself".

It's a good idea, but now your product is twice as expensive as the other guy's, and you're not going to win any bids that way, and now you're out of business.

[MilStdJunkie]

It feels like this wouldn't be all that hard to keep out of deployment with a githook and a more-or-less simple regex.

I did something very similar for a publishing system, where, well, long story short, but equivalent of local variables[1] were verboten in the production repo. On the other hand, the writers were constantly asking me to override the precommits, so, well, there's the muppet argument.

[1] https://docs.asciidoctor.org/asciidoc/latest/attributes/cust... Basically, declaring an attribute in an include target. Variable scope is one of those things getting debated in the Asciidoc world these days. Ha ha ha welcome to the 1997 SGML technical steering group, suckas. You're discovering why HTML doesn't have transclusion.

[totallywrong]

I work mostly on Infra / DevOps. I'm constantly amazed by the absurdly low standards most devs have when it comes to security. The path of least resistance is chosen 99% of time. You're definitely the exception.

[mejutoco]

IMO often this just reflects the priorities of the organization.

While it is the responsibility of devs, some system needs to be put in place to actually enforce it. Like, do not have an nfs shared volume, or incentivise anybody to report these, and give incentives for it. Otherwise "just be very careful" advice slows development to a halt.

[totallywrong]

I only half agree. Of course the company needs to have processes in place. But having shared storage can serve a multitude of legit purposes, and I feel like some alarm should go off in a developer's mind when he thinks of just dumping secrets there. Or the next one who comes and use them.

Or you might not have shared storage, and then they'll just put the creds in a Google spreadsheet like I've seen very recently.

[qingcharles]

I rooted a major web hosting provider back in the early 2000s by uploading a web shell as a profile pic (they weren't checking pictures were image files).

Once I was in it didn't take long from rummaging around in the files to first find the database credentials in a config file, then eventually finding the root password to their servers, which in fact was simply "internet" o_O

I was a nice guy so I sent them an email with their passwords and told them they might want to upgrade their forum software.

[ahhfgshando6698]

It's nfs so I always assume corporate acceptance of Microsoft is the root cause. Old companies like this are often incredibly insecure due primarily to the wildly irresponsible practice of having everything on active directory and network drives in general.

Once you've got a whole company that relies on dumping stuff on a network drive you're pretty much fucked, it's very difficult to get non-technical users to switch to SFTP. It's like pulling teeth.