[lq9AJ8yrfs]

As a sometimes peripheral and sometimes primary program manager for vulnerability disclosure, for companies you nearly can't avoid, $0.02 follows.

It's a signal vs noise thing. Most of the grief is caused by bottom feeders shoveling anything they can squint at and call a vulnerability and asking for money. Maybe once a month someone would run a free tool and blindly send snippets of the output promising the rest in exchange for payment. Or emailing the CFO and the General Counsel after being politely reminded to come back with high quality information, and then ignored until they do.

Your report on the other hand was high quality. I read all the reports that came my way, and good ones were fast tracked for fixes. I'd fix or mitigate them immediately if I had a way to do so without stopping business, and I'd go to the CISO, CTO, and the corresponding engineering manager if it mattered enough for immediate response.

[thierrydamiba]

Isn’t this a great use of llms?

Clone the repo in a sandbox and have the llm identify if the issues are real and the appropriate response based on severity level.

Wouldn’t be perfect but would have caught something like this.

[lq9AJ8yrfs]

I don't think I've met an llm that is adversary resistant, and here are counterparties that are actively playing the field, to put it mildly.

The bug bounty service providers did an adequate job of filtering out junk reports. There was a survivorship bias, some of the bogus ones that got through had an uncanny ability to twist words.

[joatmon-snoo]

Humans + LLMs are really good at producing enough spam to overwhelm anything like this. There’s a reason curl bans LLM slop reports now.