[firesteelrain]

The tools are expensive. One of the major players in the market have really expensive licensing fees. Then the developers all need to be trained on how to use the tools and understand the results. It’s not something they teach effectively in schools.

Software engineering is still kind of new overall.

[akerl_]

Apple has a massive information security organization that has pretty intense resources at their disposal.

It seems borderline impossible that there's a tool that they feel would be beneficial but that they're classed out of using by license costs or by staff proficiency.

[kimixa]

I find this amusing as Apple were the people I had direct interactions with that didn't run stuff like fuzzers or sanitizers as a matter of course - at least not in the situations I was involved with.

Though this was ~3 years ago now, and a lot of things have changed, but these tools were very much available and well known then - they aren't new. Though perhaps as they "knew" the project was coming to a close it wasn't a priority either?

It also might have fallen through the gaps due to the Apple internal/team culture - I worked for an external vendor, and we had to work against binary built framework dumps that didn't even allow us to enable things like address sanitizer completely either, and fuzzing difficult as you'd need to trace things through their opaque binary layers before it even reached our code.

Apple did have all our code though, it was very much an asymmetrical relationship, but if they were running such things as a matter of course in CI or similar you'd see that pattern in when they reported issues it caught, and the timings from time-of-bug-caused to time-of-report. It instead suggested any such runs were piecemeal and sporadic at best.

Though, it wouldn't really surprise me if they were being run and finding issues all the time, but they never actually got back to us. This certainly wouldn't be the first time we ran into "difficulties" due to the nature of the relationship and culture.

[firesteelrain]

It happens at a lot of places that the budget isn’t unlimited when it comes to information security. But even then it comes down to risk management.

[akerl_]

We’re in a thread about an Apple vulnerability, where the claim was made that they’d have found these if they’d properly run traditional tooling.

[tptacek]

Which tool specifically are you thinking of that might have found this but wasn't run because of it's very high licensing fees? I work in this field, I'll be familiar with it.

[firesteelrain]

Black Duck products

https://www.blackduck.com/fuzz-testing.html

OpenText products

https://www.opentext.com/products/dynamic-application-securi...

I won’t say how much they are here but they are very expensive.

[tptacek]

Just to be clear: your claim is that the Black Duck fuzzer would have enabled the rapid discovery of kernel vulnerabilities in macOS?

[firesteelrain]

Question was about high licensing fees and which tools I was referring to

I’m not claiming Defensics or OpenText DAST tools are magical “find all kernel vulns” buttons

My point is more that mature fuzzing ecosystems already existed before the recent AI-driven approaches. Protocol fuzzers, syscall fuzzers, coverage-guided fuzzers, sanitizers, dynamic analysis, etc. have all historically found serious kernel bugs

[tptacek]

We might just be talking past each other. My question, from upthread, is this: the heyday of AFL was over a decade ago. Every major platform company fuzzes at a scale that I think is difficult for lay practitioners to get their heads around. They contract, quarterly, soup-to-nuts assessments from competing software security companies, who get full source access and are measured against each other by the quality of their findings. They run bounty programs specifically to direct public researcher attention to these exact findings.

Why didn't "mature fuzzing ecosystems" find the vulnerabilities AI is now finding? It's a pretty big gap in the "fuzzing tools already do this" logic!

[firesteelrain]

> Why didn't "mature fuzzing ecosystems" find the vulnerabilities AI is now finding? It's a pretty big gap in the "fuzzing tools already do this" logic!

Because they simply aren’t ran. That’s my entire argument