[borski]

Ha, good to see you here.

I'm not suggesting that there is absolutely no value to any of PCI. The fact that it forces you to think about security at all is already of some value. However, I am saying that passing a PCI audit is incredibly easy as compared to thorough automated testing, and especially compared to a (good) manual penetration test. Because you can pass a PCI audit relatively easily, people will do that and think it's enough, when in reality there is far more they should be doing to protect their customers.

Should you not do PCI? No, of course you should, as it's required by the processors. Is PCI going to protect you from getting breached? No, it's not enough, and you shouldn't pretend it is.

If you have to pick exactly /one/ thing to do in addition to (or instead of) PCI, building thorough automated security testing into your SDLC process is it.

[akshatpradhan]

>If you have to pick exactly /one/ thing to do in addition to (or instead of) PCI, building thorough automated security testing into your SDLC process is it.

I don't understand how SDLC secure testing is an addition to PCI when its really a sub requirement of PCI (Req 6 which addresses SDLC and secure code testing)

I'm going to rephrase because I'm still confused: You're saying that in addition to doing PCI, I should do a sub requirement of PCI.

Why does that sound like circular logic to me?

[borski]

Show me where it suggests automated testing as a regular part of your SDLC. It recommends applying patches, coding to secure guidelines / best practices, doing code reviews, and running an automated or manual pen test at least once a year. Nowhere does it state in Requirement 6 that you should build automated testing into your SDLC.

Reference: https://www.pcisecuritystandards.org/documents/pci_dss_v2.pd...

[akshatpradhan]

Just so we're clear, what do you define as automated testing? Let me know.

[borski]

Scheduled testing that happens on an automatic basis, or that occurs whenever code is deployed or committed. For example, whenever you would run your unit tests or integration tests, you should also run security tests.

It's the difference between doing an automated or manual penetration test every 12 months and testing your application for vulnerabilities with every deploy.

[akshatpradhan]

It's 6.5.1-6.5.10 of PCI v3.1. That's all security testing. You can use a static analyzer like Brakeman to speed things along instead of doing it manually.

[borski]

Those requirements [1] state the following:

 Train developers in secure coding techniques, including how to avoid common coding vulnerabilities, and understanding how sensitive data is handled in memory.

 Develop applications based on secure coding guidelines.

It says nothing about automated testing, which is precisely my point. The requirement is that you attempt to follow security best practices and train your employees well. My point is that even the best-trained employees make mistakes.

The reason unit tests exist is to make sure you didn't accidentally break stuff when you write code. I'm arguing that automated security tests should exist for the same reason, and that's what we're trying to offer and build.

Incidentally, section 6.6 does state that you should use manual or automated testing at least annually; PCI 3 added a new clause which states 'after any changes.' Also, it explicitly specifies that you need /either/ automated or manual testing, /or/ a WAF. WAFs, as you're likely well aware, miss things very often. [2] WAFs are a good stopgap, but should not be relied upon to provide actual security; they should only be relied upon to attempt to prevent an attack while you are in the process of fixing the vulnerabilities underneath.

Because penetration testing is so expensive, typically, I suspect it will be more common for people to go with the WAF than to do a pentest with every deploy. I don't have stats to back that up, since PCI 3.1 hasn't actually been enforced yet, but I strongly suspect that will be true.

[1] https://www.pcisecuritystandards.org/documents/PCI_DSS_v3-1....

[2] http://www.slideshare.net/zeroscience/cloudflare-vs-incapsul...