[burntsushi]

So let me get this straight, you're saying that:

1. You acknowledge others' reports that 70% of exploitable bugs are rooted in memory safety problems.

2. You acknowledge that Rust helps reduce the number of memory safety bugs.

3. You resist any conclusion that Rust therefore reduces the number of exploitable bugs.

4. The stated reason for such resistance is "but but but they might exist and you might not know about them and something something something about how analyzers have gotten better."

1+2 alone seem like a pretty clear open & shut case to me. Your (4) looks like grasping at straw to me. You complain about "without evidence," but 1+2 looks like pretty compelling evidence to me.

One also has to wonder what kind of evidence would meet your standard. What evidence would convince you? And is that evidence even obtainable?

[pclmulqdq]

(1) is wrong. I said that 70% of reported exploitable bugs are rooted in memory safety problems. Not 70% of exploitable bugs. There is a big difference. In other words, I am suggesting that we have found a lot more of the exploitable memory safety bugs than other classes of bugs, which skews the number high, and that there are a tremendous number of latent exploitable bugs that are not related to memory safety that we haven't found out how to find yet (in a systematic way). I am also asserting that the Rust borrow checker likely won't help with those latent bugs.

The evidence that would convince me is ~10 years of a notable reduction in both total CVEs and total monetary value of exploits of that software (or a comparable piece of software). There is almost nothing that would convince me today that Rust is inherently more secure than C or C++ (particularly modern C++), except in the small class of applications that rely solely on memory safety for security and cannot use a garbage collector.

By the way, I do have an application running in production today that fits that requirement, and I wrote it in Rust.

[burntsushi]

Total CVEs is a poor metric given that we (Rust programmers) tend to file CVEs far more liberally than C or C++ programmers. More to the point, the evidence you claim to require is just that: reported exploitable bugs. Which seems to have the same problems as the evidence that has already been presented to you.

> I am also asserting that the Rust borrow checker likely won't help with those latent bugs.

Kind of a silly assertion, no? Particularly given you've failed to characterize these "latent bugs" other than the fact that they aren't related to memory safety. You've also failed to provide any compelling commentary regarding whether these "latent bugs" are present in C or C++ programs. If you believe they are, then Rust still makes the situation better by reducing the number of exploitable bugs by reducing the number of memory safety bugs.

Methinks you are suffering to an appeal to ridicule. Your comments are written as if someone is saying "Rust will eliminate exploitable bugs." Nobody has said that in this thread. More to the point, I'd be willing to wager that nobody with any modicum of credibility has ever said that.

[pclmulqdq]

Wait, so reporting bias in CVEs makes my proposed experiment bad, but your assertion about 70% of CVEs okay? Surely all the memory safety bugs that show up in the next 10 years will make this no contest...

Onto latent bugs: part of the idea of latent bugs is that they are latent, as in we don't know what they are yet. I am just pretty darn sure that we haven't found all the bugs in computer software, or even all the possible types of bugs. Spectre didn't show up until 2018, and it was a totally new type of bug.

I am pretty convinced, after working on huge-scale systems that get fuzzed really well by their clients, that we have found less than half of the exploitable bugs in Linux (and other similar systems). I'm not suggesting that Rust will hurt here, just that there's no evidence that it will. They will probably show up in C and C++ code too, but you never know.

By the way, one class of vulnerability that Rust code tends to introduce when compared to C++ code is supply chain attacks. Cargo/NPM/Pip are attack surfaces. So far, the worst version of this that I have heard about in Cargo was that several packages started printing some harmless politically-motivated text on startup.

Also, let's get this straight: I am saying that the claim "Rust reduces exploitable bugs" has not been proven yet. Let's stick to that, shall we?

[burntsushi]

> Also, let's get this straight: I am saying that the claim "Rust reduces exploitable bugs" has not been proven yet. Let's stick to that, shall we?

I don't know how one "proves" that claim. But it definitely has enough supporting evidence to be a very reasonable belief. I certainly believe it.

I just find your entire approach here to be so strange. On the one hand, you're demanding concrete evidence, but on the other, you're waving your hands about latent bugs and quantifying the number of known bugs with zero supporting evidence.

I think this is one of those things that's common sense and you're overthinking it.

[ducktective]

I think it's a given that idiomatic Rust code has less memory bugs than run-of-the-mill C code but IMO, this debate boils down to this question:

"Do static analyzers like cppcheck, Frama-C, Coverity or NASA's ikos catch more bugs (including memory bugs) than idiomatic Rust code?"

We have to consider the added complexity of the Rust language and the massive man-hours required to re-write old C code-bases. It's a trade-off.

[galangalalgol]

From experience, no. At least cppcheck, clang tidy, and coverity together still miss aome fairly simple use-after-free on the stack. Asan or valgrind can catch it, as long as you have a unit test that would cause it, which might require 100% code and branch coverage. In more complicated cases where it somehow dependsnon inputs, even that might not cut it.

[KsassPeuk]

Speaking about tools like Frama-C, it is important to note that while there is an overhead during development of Rust programs because of the type system, proving the absence of runtime errors in industrial size programs using Frama-C is not something easy, and it remains quite costly (and to be fair, far more costly than making Rust programs type). However, it is true that you can catch some errors that are not caught by the Rust type system, say for example, integer overflows. But I would not bet that all problems caugth thanks to the type system of Rust can be caught using Frama-C.

Now, if we focus on functional properties (proving that the code is correct), one terribly hard problem when dealing with real world programs is handling the shape of the memory. That can make some proofs awful and really hard to complete. In a language like Rust, you can get a lot of information about the shape of the memory and the memory separation for free thanks to the type system. That would make proofs really easier (this is for example what is done, but with far less precision than what the Rust type system could provide, in the different memory models configuration in Frama-C/WP and it can already dramatically improve proof performance).

It probably does not justify rewrite everything, nor changing verification tools chains that are in use in some critical domains. However, I would definitely love working on and working with a Frama-Rust tool ;)

[burntsushi]

Yes there are many trade offs to consider and context matters. But that is not the conversation that pclmulqdq is having.