Hacker News new | ask | show | jobs
by Dylan16807 2611 days ago
> As soon as a pointer to said memory is passed to an extern function in another translation unit, the compiler can't prove anything about how it's used, which is the case in pretty much all of the examples mentioned in this thread.

You would have to call such a function between the memset and the first time you write to a member. Otherwise the compiler is allowed to say "I put the padding back, and you can't prove otherwise".

> Just because reading the padding of some struct is undefined behaviour doesn't mean accessing those bits by some other means is also undefined.

It's not always undefined, but it says very clearly that the value of padding becomes unspecified.

> they almost always pass a reference to a routine in another translation unit

> you're massively overstating the relevance of compilers eliminating dead stores done via memset

Unless inlining happened, or link-time optimization, or, or...

If the compiler zeroes the memory most of the time, that makes it even scarier. Because all your tests come back clean and safe, then four years later a macro changes and suddenly you're leaking data all over the place.

I don't think I'm overstating the relevance at all. Any security feature that could disappear because of a reasonable, trying-to-help optimization is one that should have a bright red warning label. And this is such a feature. It doesn't require a "sufficiently smart" compiler, and it doesn't require a malicious compiler. This is the kind of thing that can break by accident and ruin everyone's month.

1 comments

> This is the kind of thing that can break by accident and ruin everyone's month.

Not in practice. Compilers make use of undefined behaviour to optimize things that are widely applicable and profitable. No real compiler does what you're saying and no future compiler is likely to without explicitly being asked to.

I agree that, by the letter of the spec, you're right, but you're still most certainly overstating the relevance.

I'm not arguing that this isn't a real problem or that people shouldn't use memzero_explicit() (or similar) where security is on the line, as I already said several times in another sub-thread -- I'm just saying that this kind of thing is extreme language-lawyering beyond the realms of probability. It's still not an excuse to be lax, but let's be realistic about the actual likelihood of it happening.

It's just a type of dead store elimination. And objects are initialized so often that I could easily see it happening in the future. But I guess we'll just disagree on how likely it is.
> It's just a type of dead store elimination

It's not just any, typical kind of dead store elimination. It also would require other optimization passes that I can assure you no mainstream compiler actually does. You can disagree with me if you want, but you're simply wrong.

Lots of mainstream compilers already have passes that check if every field in an object is definitely assigned. They use this to provide errors or warnings.

That step is 90% of the work. Once you can do that, it's straightforward to assess that every field is assigned after a memset, with no intervening reads, and then remove the memset.

> That step is 90% of the work

I spent several years working on a production grade compiler and I can assure you it's not. But keep just making things up off the top of your head if it makes you feel smart.

Would you care to explain why?

Let's look at a basic common case, as a checklist.

1. All fields are definitely assigned.

2. No functions are called except intrisics.

3. Nothing reads from the object on any control path.

4. Memset happens between creation and first assignment.

We agreed that step 1 is a solved problem, right? Are any of 2-4 difficult? Did I miss any prerequisites for the optimization?

Once you can prove 1-4, isn't the optimization pass as simple as looking for memsets applied to structs, checking 1-4, then deleting the call?