[tiraniddo]

You can bet I'm salty. I do Windows research and I am a owner of the Chromium Windows sandbox code so I have a vested interest in this.

The problem with dealing with Windows and by extension Microsoft is there's no way of inspecting what they've changed from version to version other than by RE or having test cases. However, we try and avoid writing unit tests for behaviors Microsoft's responsible for, but of course in many cases MS don't have those tests either so these things fall through the cracks.

[mehrdadn]

I have a question. With the sheer number of Windows components each having objects whose security and inheritability can be manipulated individually, and with tokens and privileges controlling access on top of that, and with so many users and groups to account for, and with so many special cases for consoles etc. to consider, I'm surprised at how few of these bugs there seem to be. It seems like a herculean feat to track everything for even just a single version of the kernel, let alone in an ever-evolving entire OS. Do you know how Microsoft keeps track of all the security barriers, interactions, etc. in the entire OS? Do they have entire teams dedicated to just tracking the security privileges/permissions/etc. for all the components?

[amenod]

> However, we try and avoid writing unit tests for behaviors Microsoft's responsible for,...

When you say "unit tests", this makes sense. But wouldn't it be wise to have integration tests in place that would guard against such regressions, either in your code or Microsoft's?

[tiraniddo]

In Chromium we do have integration tests for the sandbox functionality as a whole and unit testing but it doesn't cover things like this as we're testing Chromium's ability to sandbox not whether the OS's primitives have broken. We might notice if all of a sudden our sandbox stopped working, but for something which only exhibits a problem when it's being actively circumvented we won't.

I can't speak to what MS do testing wise, considering the age of some of this code it seems likely there's no test for this specific functionality otherwise you'd assume it would have been noticed. Testing for security defects is inherently difficult anyway, especially logical flaws where you don't get a nice crash. This case is different but in general you usually need some specific setup process to get the system into a vulnerable state which is hard to achieve without knowing ahead of time the bug you were trying to detect.