[mre]

> However, their use is the exception, not the rule, and their use—in particular in combination—requires security expertise and investment that is not common. For them to provide real-world, large-scale improvements in the security outcomes of using C and C++ software, there remains significant work to be done. In particular, to provide security benefits at scale, for most software, these protections must be made an integral, easy-to-use part of the world-wide software development lifecycle. This is a big change and will require a team effort.

That's the core problem.

The mechanisms mentioned are primarily attack detection and mitigation techniques rather than prevention mechanisms. Bugs can't be exploited as easily, but they still exist in the codebase. We're essentially continuing to ship faulty software while hoping that tooling will protect us from the worst consequences.

Couldn't one argue that containers and virtual machines also protect us from exploiting some of these memory safety bugs? They provide isolation boundaries that limit the impact of exploits, yet we still consider them insufficient alone.

It's definitely a step in the right direction, though.

The paper mentions Rust, so I wanted to highlight a few reasons why we still need it for people who might mistakenly think this approach makes Rust unnecessary:

  - Rust's ownership system prevents memory safety issues at compile time rather than trying to mitigate their effects at runtime  
  - Rust completely eliminates null pointer dereferencing  
  - Rust prevents data races in concurrent code, which the paper's approach doesn't address at all  
  - Automatic bounds checking for all array and collection accesses prevent buffer overflows by design  
  - Lifetimes ensure pointers are never dangling, unlike the paper's approach which merely tries to make dangling pointers harder to exploit

So, we still need Rust, and we should continue migrating more code to it (and similar languages that might emerge in the future). The big idea is to shift bug detection to the left: from production to development.

[commandersaki]

We're essentially continuing to ship faulty software while hoping that tooling will protect us from the worst consequences.

Yet one way to measure how effective these mitigations and countermeasures are working is looking at the cost of the zero day market. The trend continues to going upwards in the stupidly expensive realm due to needing multiple chains and such to attack software. However, I'm not discounting software now developed in memory safe language doesn't already contribute to this.

Here is one of the references indicating this in the article: https://techcrunch.com/2024/04/06/price-of-zero-day-exploits...

[tgv]

Or Java, or Scala, or Go, or whathaveyou. This is about existing software.

[unscaled]

While all of the languages you mention are memory safe (as is almost every programming language released after 1990), none of them solve all of the safety problems mentioned above, in particular, the two points:

  - Rust completely eliminates null pointer dereferencing  
  - Rust prevents data races in concurrent code, which the paper's approach doesn't address at all  

Scala comes closest to solving these points, since it has optional features (in Scala 3) to enable null safety or you could build some castles in the sky (with Scala 2) to avoid using null and make NPEs more unlikely. The same goes for concurrency bugs: you can use alternative concurrency models that make data races harder (e.g. encapsulate all your state inside Akka actors).

With Go and Java there is no dice. These languages lack the expressive power (by design! since they are touted as "simple" languages) to do anything that will greatly reduce these types of bugs, without resorting external tools (e.g. Java static analyzers + annotation or race condition checkers).

In short, Rust is one of the only mainstream languages that absolutely guarantees safety from race conditions, and complete null safety (most other languages that provide good null safety mechanisms like C#, Kotlin and TypeScript are unsound due to reliance on underlying legacy platforms).

[tgv]

Nil dereferencing in those languages doesn’t make them unsafe. It throws an exception or panics. And Java has some none-nil annotation, IIRC.

Still, none of this is relevant to existing software written in C. This is not about a rewrite.

And if it were, rust doesn’t offer perfect safety, as many tasks almost demand unsafe code. Whereas that doesn’t happen in Go, Scala, etc. Every situation requires its own approach.

[eklavya]

How is NPE a safety issue?

[pron]

> We're essentially continuing to ship faulty software while hoping that tooling will protect us from the worst consequences.

It is the consequences that give rise to the cost of a bug or the value of preventing it.

> Couldn't one argue that containers and virtual machines also protect us from exploiting some of these memory safety bugs? They provide isolation boundaries that limit the impact of exploits, yet we still consider them insufficient alone.

No, that's not the same because sandboxing only limits the impact of vulnerabilities to whatever the program is allowed to do in the first place, and that is, indeed, insufficient. The mechanisms here reduce the impact of vulnerabilities to less than what the program is allowed to do. To what extent they succeed is another matter, but the two are not at all comparable.

> The big idea is to shift bug detection to the left: from production to development.

This has been the idea behind automated tests since the practice first gained popularity. But it's important to understand that it works due to a complicated calculus of costs. In principle, there are ways to eliminate virtually all bugs with various formal methods, yet no one is proposing that this should be the primary approach in most situations because despite "shifting left" it is not cost effective.

Everyone may pick their language based on their aesthetic preference and attraction to certain features, but we should avoid sweeping statements about software correctness and the best means to achieve it. Once there are costs involved (and all languages that prevent certain classes of bugs exact some cost) the cost/benefit calculus becomes very complex, with lots of variables. Practical software correctness is an extremely complicated topic, and it's rare we can make sweeping universal statements. What's important is to obtain empirical data and study it carefully.

For example, in the 1970s, the prediction by the relevant experts was that software would not scale without formal proof. Twenty years later, those predictions were proven wrong [1] as unsound (i.e. without absolute guarantees) software development techniques, such as code review and automated tests, proved far more effective than anticipated, while sound techniques proved much harder to scale beyond a relatively narrow class of program properties.

Note that this paper also makes some empirical claims without much evidence, so I take its claims about the effectiveness of these approaches with the same scepticism as I do the claims about the effectiveness of Rust's approach.

[1]: https://6826.csail.mit.edu/2020/papers/noproof.pdf

[sramsay]

> Everyone may pick their language based on their aesthetic preference and attraction to certain features, but we should avoid sweeping statements about software correctness and the best means to achieve it. Once there are costs involved (and all languages that prevent certain classes of bugs exact some cost) the cost/benefit calculus becomes very complex, with lots of variables. Practical software correctness is an extremely complicated topic, and it's rare we can make sweeping universal statements.

Thank you. I feel like this perspective is forever being lost in these discussions -- as if gaining the highest possible level of assurance with respect to security in a critical system were a simple matter of choosing a "safe language" or flipping some switch. Or conversely, avoiding languages that are "unsafe."

It is never this simple. Never. And when engineers start talking this way in particular circumstances, I begin to wonder if they really understand the problem at hand.

[kobebrookskC3]

does making software scale? i see exploits for android/ios even though they spend millions if not billions on securing it. which unsound techniques make exploits unfeasible? i'm not even looking for a guarantee, just an absence in practice.

[pron]

Well, software today is much bigger and of higher quality that was thought possible in the seventies. That's not to say that it can scale indefinitely, but my point was that unsound methodologies (i.e. not formal proofs) work much better than expected, and the software correctness world has moved from the seventies' "soundness is the only way" to "software correctness is a complex game of costs and benefits, a combination of sound and unsound techniques is needed, and we don't know of an approach that is universally better than others; there are too many variables".