| I'm sorry that's been your experience. What should have happened is a primarily manual penetration test, administered by security engineers who themselves used to be fully competent software developers. Any automated tooling should have had strict manual verification and should not have been the focus of the test. Furthermore, superfluous results should not have been submitted in the PDF report. Strictly "policy" audits such as PCI compliance differ a bit, but in general they should still involve a technical deep dive into your product's infrastructure, conducted by consultants with expertise in multiple tech stacks and overall experience in a variety of frontend and backend frameworks. The final deliverable ("PDF report") also should have been hand-written, and in language that conveys technical expertise, complete with recommended steps towards remediation of any issues. My employer, Accuvant does this, as well as Matasano (more well known here on HN). As for why it's so expensive...well, I bill out at about $2000 per day. It really comes down to a lot of what people like patio11 and tptacek like to talk about here regarding consulting: 1. This is highly specialized work, with a much smaller population of competent engineers than typical web developers (for example). As such, it naturally receives a higher fee for supply and demand. Now, some people abuse this and run scans like Nessus and call it a day. These are not real infosec firms, they are parasites. 2. More specifically, we ask for it and we receive it, and we do exceedingly well. If people keep paying us five figures a week to perform a penetration test, we're not going to stop asking for it or reduce our prices. |
The entire consulting penetration testing market is setup to encourage this behavior. There is no way to prove you actually did anything correct. Someone can write a wonderful PDF analysis by hand and still leave the system full of glaring holes. Customers can't tell a system is broken until it gets hacked.
>More specifically, we ask for it and we receive it, and we do exceedingly well. If people keep paying us five figures a week to perform a penetration test, we're not going to stop asking for it or reduce our prices.
Right, but many times I've seen companies do it because they are desperate to do it for compliance purposes. :/ Essentially there is a non-trivial portion of the market held up by regulatory demand.
>Strictly "policy" audits such as PCI compliance differ a bit, but in general they should still involve a technical deep dive into your product's infrastructure, conducted by consultants with expertise in multiple tech stacks and overall experience in a variety of frontend and backend frameworks.
I'm curious. Do you review every line of code in a customer's codebase? What about the code of every library they import? If you don't review imports, do you leave a big caveat in your report that says their code looks okay, but the libraries could be full of vulnerabilities?