| Agreed. It is difficult to properly set expectations for assessment results due to the cultural demand for a clean bill of health. No one wants to be sold "meaningful insight" into their security posture, they want to be sold a report that says no vulns are present after they're fixed. Fundamentally, I believe the security consulting industry is due for a radical shift, probably instigated and led by Hackerone and Bugcrowd. Unfortunately there is a lot of inefficiency in the industry that allows consulting firms to exist as they do now. For the most part, my clients come to me for an assessment because they have a measurable business need - lucrative customer A is demanding an external third party assessment. This is the primary use case for which I feel comfortable - my time at Accuvant (now Optiv) left me deeply uncomfortable with the rote way that security assessments could be nosebleed expensive for frankly questionable work (e.g. $10k/week/assessment for reviewing brochure websites for large companies - for the most part employees knew what they were doing, it was just overpriced and unnecessary). In a lot of ways security assessments are inflated in price because they're somewhat like insurance. Truly exceptional vulnerability researchers could and probably should be earning half a million to a million a year. Watching them work is a beautiful blend of art and science. They are underpaid. On the other hand, merely competent or outright mediocre "penetration testers" are overpaid by way of de facto rent collecting. If I were to run a productized software firm now and no particular customer demanded a third party assessment, I'd honestly never commission one. Instead, I'd open a bug bounty program and dial the rewards up, then welcome specific people to come find vulnerabilities (people like Frans Rosen of Detectify, Jack Witton of Facebook, Egor Homakov a competitor of mine and in this very thread ;) or Bitquark at Tesla - not sure of his real name off the top of my head). I have utter confidence that for essentially everything but cryptanalysis, a generously priced bug bounty is plainly superior than any given firm's commissioned assessment in raw results. It's not quite as turnkey or comforting, but it's effective. Hackerone and Bugcrowd even field reports for managed programs these days. I believe this wholeheartedly enough that I would (and have in the past!) advise potential new clients against the interests of my firm in this direction if they didn't require the assessment for an external third party or regulatory compliance. Once they really perfect the researcher signal/noise rating system, Hackerone and Bugcrowd are going to take the top 100-1000 researchers on either platform and wrap their current activities into a neat layer of turnkey abstraction, call it a formal assessment and legitimately disrupt the pricing of the security consulting industry. |