[flowerbeater]

This is a reasonable and balanced analysis of the situation. In retrospect, it seems like the reversion of the 190 patches was an overreaction that ended up causing a lot of confusion: many people even on HN misinterpreted the comments on reversions to believe that bad patches were committed to the source tree or to stable.

But besides the lesson that one ought not to be deceptive with submitting patches, is also the lesson that the kernel is not as well reviewed as one may hope and with some effort, it's certainly possible to add an undetected vulnerability. I think that's probably one thing that led to the drama, is that the fundamental trust and work of the kernel was attacked, and the maintainers felt the need to fight back to protect their reputation.

[tytso]

190 patches were not reverted. 190 patches were proposed to be reverted, and those reverts have been going through the normal kernel review process. In many cases, the patches were confirmed to be correct, and so the revert was dropped. In 42 cases, those commits were found to be inadequate in some way; in some cases, the commit didn't actually fix the problem, or introduced another problem, or I believe in one case, actually introduced a security problem(!). Call those last set of patches, "good faith hypocrite commits".

Remember, too, that many of these commmits were in random device drivers. Consider how often random Windows device drivers crash or cause random blue screens of death. Not all "SECURITY BUGS (OMG!)" are created equal. If it's in some obscure TV digitalization card in the media subsystem, it won't affect most systems. Core kernel subsystems tend to have much more careful review. As the ext4 maintainer, I'm a bit slow in accepting patches, because I want to be super careful. Very often, I'll apply the patch, and then looking at the changed code in the context of the entire source file before approving the commit. Just looking at the diff might not be enough to catch more subtle problems, especially dealing with error handling code paths.

The problem with device drivers is that in some cases, they are written by the hardware engineers that created the hardware, and then as soon as the hardware ships, the engineers are reassigned to other teams, and the device driver is effectively no longer maintained. One of the reasons why some maintainers are super picky about allowing device drivers to be admitted to the kernel is because the concern that driver author will be disappear after the patch is accepted, and that means the subsystem maintainer is now responsible for maintaining the driver. So if you want to sneak a SECURITY BUG (OMG!) into the kernel, targetting some obscure device driver is the going to be your simplest path. But that's really only useful if you are gunning for a IEEE S&P paper. The obscure device is not likely to be generally used, so it's not that useful.

(Unless, of course, you are a hacker working for Mossad, and you are targetting a super obscure device, like, say, a nuclear centrifuge in use by Iran.... in that case, that security vulnerability only works on a very small number of systems is a feature, not a bug. :-)

[andrewflnr]

You mean sometimes you don't look at the change in the context of the whole file? That's terrifying.

[ashkulz]

I assume it depends on the nature of the diff? I do a lot of code review at $JOB, and sometimes the diff is so obvious that there's no need to look further.

OTOH if the code is something that I've haven't looked at in a while or don't understand that much, I'll read around a bit and see if there's a way the diff can be improved.

[detaro]

I don't think the latter part is true, my impression is that the kernel people are very well aware of the limits of their review ability and don't pretend to be unfoolable.

[flowerbeater]

There's a wide range of degrees between "unfoolable" and "can be done by a persistent student". I think the impression (at least my impression) used to be is that it was possible before but quite unlikely without state-level efforts, but now we understand a properly advised student can get most of their attempted vulnerabilities inserted.

[detaro]

The pure number of just regular bugs that aren't caught is already a good indicator that not much special effort is needed. (And "just a persistent student" isn't that little, given that the group also contributed regularly to the kernel, was studying its security, ... and thus quite familiar with the field, and the kind of people a nation state would employ for that)

[Mathnerd314]

The harder part is probably fooling all of the static analysis tools that get run on the kernel, Coverity, Coccinelle, Smatch, and so forth.