[zamadatix]

If the people you're paying to find weaknesses in the security system are assuredly never going to find a way to access internal data then how did you conclude you needed a pen tester in the first place? I mean, it's probably the right conclusion but only precisely because they'd find a way to access things they shouldn't be able to.

[debo_]

It's relatively common to have pen testers attack a cloned environment w/ sanitized data. This is especially true in cases where your policies (or those you've agreed to from customers) require you to present evidence that you are having a pen test done every X years.

[red-iron-pine]

access to live data for testing is also a compliance question -- as in, don't do it, and why are you doing it?

why are you not using cloned or dummy data?

[x0x0]

We spin up a clone of prod and point them at that.

Certainly if a weakness is found in the clone it's also present in prod, but that's what contracts are for. And we also review logs to make sure.

edit: a clone of prod w/ only test data in it, not prod data.

[randomdata]

How do you know what you are looking for in the logs?

If you have the foresight to be able to recognize a malicious action from the logs, why not have the software block those actions from the start?

[x0x0]

We log all accesses and flows. So eg if our pentesters found a vulnerability in an endpoint, we can retrieve every post against that endpoint and (1) verify the pentesters didn't exploit it against prod, and (2) verify that it hasn't been exploited by anyone else.

[randomdata]

Of course, that only works if the vulnerability is reported. There is no reason for the malicious actor to report the vulnerability they have chosen to exploit.

What percentage of the vulnerabilities discovered are independently discovered by multiple pen testers?

[unethical_ban]

It sounds like you're suggesting that pen testers by default will not reveal discovered vulnerabilities with clients.

Then you talk about "discovered and revealed vulnerabilities". But, your first sentence talks about "discovered vulnerabilities not revealed".

What you may be wanting is a honeypot, where a pentest client intentionally puts some vulnerabilities of various exploit difficulty into the clone environment to ensure pentesters are doing their job.

[randomdata]

> It sounds like you're suggesting that pen testers by default will not reveal discovered vulnerabilities with clients.

How so? Presumably most pen testers are working in good faith. But, if there is a malicious actor in their midst, that individual would not disclose any vulnerabilities they intend to exploit, no. What would be the point? That's just a really good way to get caught.

> Then you talk about "discovered and revealed vulnerabilities".

Yes, that's right. While it is theoretically possible for all your pen testers to be working together maliciously, if you are careful in your employment practices you can make this highly unlikely.

As such, if your data shows that 100% of all known vulnerabilities were independently discovered by multiple testers, then there is reasonable confidence that any malicious actor's failure to disclose a vulnerability will still be reported by someone else.

But if that figure is less than 100%, and especially if it is considerably less than 100%, then there is much more doubt cast on another pen tester in your organization's ability to find the same vulnerability. Here you have a problem.

[digital_sawzall]

>What percentage of the vulnerabilities discovered are independently discovered by multiple pen testers?

I'd warrant nearly all of them, though it may take a while.

If you have ever submitted or worked with a bug bounty program you will run into dozens of duplicates.

I've personally performed and overseen assessments in which the company had already done a complete blackbox pentest and wanted a second whitebox review to make sure the first company knew their stuff and validate they found the same bugs. Also did a few of the honeypot assessments in which companies put purposely vulnerable code in to make sure 'we are doing our job', I hate those most.

Depending on the testers speciality of course, the reports often found the same or similar issues.

Source: 15 years as a pentester, offensive security engineer, and now security architect.

[randomdata]

> I'd warrant nearly all of them, though it may take a while.

Why guess when the other commenter has the actual data...?

[x0x0]

> What percentage of the vulnerabilities discovered are independently discovered by multiple pen testers?

Zero because we patch them as soon as we are notified. Generally at the end of the test / before the retest, but if they found something serious they would notify immediately,

[randomdata]

Patch production, sure, but naturally you would leave them in the pen testing environment for some time in order to collect data. No data and you’re just guessing. That’s fine for amateur hour, but not business.

[mkii]

It could have been for a service that was not in production yet, and in an isolated environment.