[app13]

The CVE process itself is broken. HackerOne and company VDPs are inundated with new reports of varying quality thanks to the advancements (I think) in agentic AI. It's allowed for both an increase in trash-tier low quality AND legitimately high quality reports. Since the same AI's are writing both, its almost impossible to distinguish between the two at a surface level.

In response, companies just aren't responding like they used to. I spoke at a cybersecurity conference In June and the overwhelming "vibe" on the floor and in the talks was that responsible disclosure was dead or dying, and public disclosure is the way forward. The Microsoft and Nightmare Eclipse situation was oft cited.

[demosthanos]

As someone on the company response side of the HackerOne brokenness, I can confirm that this effect is real but would also note that the difficulty of distinguishing is not as severe as all that, because the companies have access to the source code which the researchers do not typically have access to.

This means that the token cost of verifying any given HackerOne report is dramatically lower than the token cost of producing a report in the first place. Automated triage systems should be possible, and realistically it's well within the capabilities of most companies to go further and actually automate the Red Team side of it and catch issues before they surface in the black box research. From what I've seen doing so should cost dramatically less in tokens than the bounty payouts do.

The problem is that security is woefully underfunded in most companies, so even an infosec organization that saw the deluge approaching from a distance may well not have had the resources to prep for it even if they knew exactly what actions they would take if they had the capacity.

[QuadmasterXLII]

The token cost of a report is lower bounded by the number of tokens in the report * price per token of the cheapest model. The token cost of a good report is much higher, but sifting out the good reports is the entire problem.

[demosthanos]

In theory, yes, but are you actually seeing clearly-garbage lower-bound reports like this?

The ones we're seeing show clear evidence of being AI-generated, are often incorrect or duplicated, but they also show clear evidence of the AI having done its homework and spent a while crawling our API.

Even if we were getting reports at the lower bound you're describing, those would be even easier to triage: just add a quick step to check if the API in question even exists, then if it does that very cheap "where is this API" query becomes part of the input to the second-level triage that spends more tokens.

[infinite_spin]

Do you have a tier based system for identifying high value reporters? Like a credit score for vulnerability hunters?

[demosthanos]

Not formally, but we do informally recognize names and prioritize researchers who reliably turn up good results.

[infinite_spin]

that might be a nice incentive for bounty hunters, a sign of recognition and a good metric for triage

[tedivm]

Their CISO literally acknowledged it and then they all continued ignoring it again. This isn't just bad process, this is a broken security organization.

[sandeepkd]

Should a company promoting the enterprise usability of AI, itself start with building a intake process to distinguish between the noise and signal for these reports. If you cant solve your own problems with your product then how do you expect the customers to be able to use it.

[pseudohadamard]

Not even that. Even before AI came along the widespread practice of CV-Enhancement was slowly strangling the reporting of actual legitimate, needs-to-be-fixed issues. When it turns into a giant shit-shovelling exercise it's not surprising that some of the shit doesn't get shovelled.

Not defending HackerOne, but pointing out that it's not a black-and-white issue.