[PaXTeam]

> The problem is that getting things into small individually testable components > is literally anathema to the Pax/grsecurity model.

no, you're wrong. how do you think we developed our code? did all the 8+ MB worth of it pop out of our head all at once? or more realistically, did we develop the features piece by piece, not unlike how upstream linux is developed?

> They have very specific "chunks" of functionality which are quite invasive, by design.

define invasive. linux itself has 'quite invasive' features too yet that didn't prevent them from being developed and upstreamed, so not sure what you were trying to imply here.

> This goes against the model the Linux kernel is developed, so the two development > communities are simply mutually exclusive.

this narrative only exists in your head, not in reality. our work is as much upstreamable as any other kernel code that went in over the years (how else do you think some of it could get in already?), it's just that it can't be done in one's free time.

[SEJeff]

Well I recall a somewhat recent case where you mentioned how there was one of features (I want to say it was the KASLR) that depended on another feature which still isn't in. You (spender in specific) mentioned on LWN how basically the feature was no good without the other 1/2 and the two changes were both somewhat invasive (in the Linux kernel context what Linus would consider invasive). I'm not saying you're wrong in any way or that you developed all 8+MB of code in 1 chunk. I'm saying you design your components to go together and depend on eachother, not to be split up individually and added sort of kind of willing/nilly.

Did I get that wrong? From reading Brad on LWN for literally years wank about Linux security (he's absolutely not wrong to complain), this was my take. Or it could be summarized as (this is satire):

    PaxTeam: This is our code, it is open source, it is more secure than the dumpsterfire that is upstream, take it if you want it.
    
    Linux Upstream: That is nice, now break it into 1000 individual patches each independently git bisect-able. Also, can you change the interfaces of these design flaws in Linux and THEN entirely re-do your well tested patch to match the way we do code upstream and match the interfaces we would *like* Linux to have that we're going to expect you to code for us.
    
    PaxTeam: No, we want to do more interesting things like build more secure patches for Linux and look at pictures of cats on the internet, as it was designed to be used for.
    
    Linux Upstream: ...

[PaXTeam]

i don't quite get what you're arguing now... are you stating that when there're functional dependencies between components, we should somehow ignore them when incorporating them into a larger piece? say, the kernel should get a NIC driver before it got a network stack at all? if you're not arguing that then i don't see why different rules should apply to the security features we have...

as for your satire, it's just got one thing wrong, but that kinda kills the rest of it: we never said "take it if you want it" and thus we never embarked on the rest of that journey.

[derefr]

> did all the 8+ MB worth of it pop out of our head all at once? or more realistically, did we develop the features piece by piece, not unlike how upstream linux is developed?

That's not an argument against the current state of the pax/grsec being much more monolithic in architecture than the Linux kernel (I don't know if it is; just that that argument doesn't prove it's not.) Sure, the commits are small. 1000 small commits can add up to 100 small+simple components, or to 10 rather large+complex ones.

> define invasive

"Invasive", I believe, refers to how much of the kernel needs to be modified to upstream any given feature (regardless of patch size.)

The opposite of "invasive" is "isolated". 1000 lines of code in one module representing a new USB device driver, for example, is very non-invasive. It doesn't even need to be "hooked into" anything; it just puts a vendor+device ID into a database, and some existing logic will then detect devices with such IDs showing up on the bus and load the module in response.

Meanwhile, 1000 lines of code that make breaking changes (however small) in the ABIs or preconditions/postconditions of 100 different existing kernel functions, is an "invasive" patch. It requires a lot more testing than the USB driver to be accepted, because those 100 existing functions will be executed by a lot more people's machines than the USB-device module (which will only get executed on machines that have that device.)

[PaXTeam]

i don't think you read carefully what i replied to so here's it again:

> The problem is that getting things into small individually testable components > is literally anathema to the Pax/grsecurity model.

nowhere does this talk about what the patch looks like. it talks about "getting things into small individually testable components" being "literally anathema to the Pax/grsecurity model". i.e., that decomposing the monolithic patch into composable pieces is somehow against our development model whereas said monolithic patch was developed exactly this way (which means the reverse process would reproduce these 'testable components'). that argument (my statement) directly contradicts Jeff's so only one of them can be true and as the author of the code in question i know which one is what ;).

as for invasiveness, you're only stating the obvious that there're more and less complex pieces of code yet both kinds (well, all kinds in the complexity spectrum really) can make it into the kernel. that is, this argument cannot be used against incorporating grsec on its own.

[derefr]

> as for invasiveness, you're only stating the obvious that there're more and less complex pieces of code yet both kinds (well, all kinds in the complexity spectrum really) can make it into the kernel. that is, this argument cannot be used against incorporating grsec on its own.

You asked for a definition, not an argument. Definitions are supposed to be obvious. Not every line of a reply has to be a rebuttal, just because one part is. :)

> which means the reverse process would reproduce these 'testable components'

The reverse process of how you built up the system (by committing changes), would break the the patchset down into commits. Commits might be patches, but they're sure not components.

A commit can change a line in the middle of a function in the middle of a random file anywhere in the kernel. A component is its own file/module/function that can be independently tested and verified and, most importantly, judged for how well the design-choices in its implementation solve the user-story they purport to solve, compared to other possible implementations.

A random one-line commit can certainly implement functionality—or can fix a bug—but it can't (usually) introduce a user-visible feature on its own. A series of one-line commits can, together, introduce a user-visible feature; but if they do, they need to be merged together into a single patch so that the design they implement can be seen, in order to be evaluated.

If you have take your codebase and just export all the commits as patches, each one might—as it probably originally did—just add one or two lines to the body of some function. But if all the commits necessary for a given feature end up combining into one big, monolithic blob of a patch—or end up extending several functions in the kernel into being large, monolithic functions—then the design of the feature is probably bad, and the patch will therefore probably be rejected.

What the kernel maintainers expect, in such a case, is for you to take your features and refactor them into independent subfeatures, such that each "patch that implements the feature" is fairly small. They expect you to do the same thing to your patchset that is done in a web-app when it is refactored into microservices: to make each component small enough, and non-interdependent-enough, that an argument over the design of the patch can involve each invested party writing their own version to demonstrate their idea; and where each component can be easily ripped out and replaced with one of the alternate designs, without affecting any of the other components.

For a good case of this already happening, look at OpenVZ. OpenVZ was a big, monolithic patch, that was never going to get upstreamed. It was refactored into the individual components of cgroups, the CPU freezer, memory quotas, and the addition of a number of namespace hierarchies. All of these components "add up to" something roughly equivalent to OpenVZ, but they're not OpenVZ. (In fact, they're effectively Docker; upstreaming them basically killed OpenVZ!)

[PaXTeam]

> 1000 small commits can add up to 100 small+simple components

vs.

> Commits might be patches, but they're sure not components.

make up your mind, are components (whatever that means for you) composed of commits or not? :) as for commits vs patches, all commits are patches + description, but we use those terms interchangebly in the linux world with this understanding. in any case, i stand by what i said, we develop our code the same way upstream linux (or really any sane project) does. we're just not developing it for the purpose of inclusion, maybe that's the source of your confusion.