[ruthmarx]

> I'm less interested in the feature sets of the two. I think what'd be more interesting is replicate exploitation scenarios with their default policies and see which subsystem succeeds in mitigating the exploit and which fail.

The feature set is exactly what dictates which systems are more likely to prevent exploitation, though.

App Armor simply isn't as granular, and simpler to bypass (e.g. by making a hardlink to a file to override AppArmor policy).

AppArmor may be good enough in many situations, but SELinux gives you much more control, so you can be much closer to perfect to protect against unknown situations.

[commandersaki]

Still don't care about the feature set, show me a useful benchmark. If SELinux prevents a hypothetical that's great, but security is a tradeoff and I'd opt for convenience and simplicity to sacrifice potentially negligible risk.

For example, I'm seeing that SELinux didn't mitigate ShellShock where AppArmor did (despite being an attack vector that isn't really common). But these are the things I want to know.

[ruthmarx]

> Still don't care about the feature set, show me a useful benchmark.

"I don't care to understand the differences in security thees systems provide"

That's what you're saying here. Which means you're not going to be evaluating these systems in any way that matters.

> If SELinux prevents a hypothetical that's great, but security is a tradeoff and I'd opt for convenience and simplicity to sacrifice potentially negligible risk.

You're asking for benchmarks, but already here your willing to dismiss the results because you don't really care about them, you care about "convenience and simplicity" and security being good enough, right?

In that case, sure AppArmor is good enough for most people.

But so is a flimsy chain look. If you want to actually secure something and not just deter, you'd want a deadbolt and sturdy door, right?

The point is, the feature sets matter, precisely because so many attacks are hypothetical. You have to see and speculate what attackers might do and have things in place to prevent that. SELinux facilitates that a lot more than AppArmor does.

That's the fundamental point here, and not something you will likely find a nice graph to support. If you ask an AI to generate one for you it might be able to though. If you really still think you need it.

> For example, I'm seeing that SELinux didn't mitigate ShellShock where AppArmor did (despite being an attack vector that isn't really common).

Well that's nonsense. SELinux can protect against anything AppArmor can since AppArmor provides only a subset of features.

I sense you might not be interested in this since you've said you just want benchmarks, but here's a page from RedHat not only explaining how SELinux prevents Shellshock from being able to do damage, but even walks you through exploiting it on an SELinux enabled system so you can test it yourself[1]. There's also a blog post from Dan Walsh explaining how SELinux constrains shell shock [2].

I'm also less confident AppArmor is as effective against container escape exploits like these [3] [4]

> But these are the things I want to know.

Mmm. Well, there's no benchmarks. But if you do research you will find the examples you want. SELinux will have substantially more examples because it's employed in wider use due to RedHat and Android.

If you really want to compare, look at the Debian and RedHat security advisories, and look how many RedHat has saying that SELinux provides protection if enabled, and how many Debian and Ubuntu have being unable to say the same for AppArmor.

But really, again, you should bother to understand the feature sets and actual technology. This approach you want looking for benchmarks, it's not going to necessary be accurate or representative, and I say that confident that your methodology would still show SELinux as the better option (in terms of security, not usability/convenience).

The key is that you should strive to understand systems you want to use, not just look for a blog article that can provide justification for an intuition or desire.

[1] https://github.com/RedHatDemos/SecurityDemos/blob/master/201...

[2] https://danwalsh.livejournal.com/71122.html

[3] https://www.redhat.com/en/blog/latest-container-exploit-runc...

[4] https://www.redhat.com/en/blog/selinux-mitigates-container-v...

[commandersaki]

> Well that's nonsense. SELinux can protect against anything AppArmor can since AppArmor provides only a subset of features.

This is from Dan Walsh who is quite famous in the SELinux community [1]. See the part "Why didn't SELinux block it?"

Sure you can configure SELinux with a stricter policy, but virtually nobody does that in practice, they use the defaults, mostly because SELinux policies are really only used to fix stuff SELinux default policies break, usually using the audit2allow or whatever its called.

> I sense you might not be interested in this since you've said you just want benchmarks

If the EDR/antivirus industry can have various test suites and testing organisations, the same methodologies can apply to OS security subsystems. Let's see what happens with your default RH/SELinux, Ubuntu/Debian & AppArmor, and for shits and giggles OpenBSD, and see how they fare against exploits and vulnerabilities harvested over the years, and make reproducible labs. That's what I'm asking; I'm not going to scour security advisories to do tit for tat comparisons, I would rather see a well thought approach to this.

We do this kind of comprehensive benchmarking with all sorts of software such as compression, cryptographic libraries, compilers / languages, etc. Traditionally this would've been harder in the days when virtualisation and utility computing was nascent, but the infrastructure part is pretty achievable these days. Just need to someone to expend the effort (and I'm not volunteering).

[1]: https://danwalsh.livejournal.com/71122.html

[ruthmarx]

> Dan Walsh who is quite famous in the SELinux community

Yes, I know who Dan Walsh is, it's part of why I linked to his article in this case.

> See the part "Why didn't SELinux block it?"

Yes....did you understand it? Or just read the title and assume you were right?

SELinux didn't prevent the exploit code from running, which is exactly normal and exactly the same as AppArmor.

What it did was prevent the code from being able to do anything.

So in that sense, SELinux absolutely stopped Shellshock to the same extent AppArmor would have.

> If the EDR/antivirus industry can have various test suites and testing organisations, the same methodologies can apply to OS security subsystems.

There's fundamentally less need because thees are open source systems and the design is sufficient to judge. You're just being incredibly lazy and justifying not wanting to learn these designs, or you otherwise don't want to put in the work to do so.

Imagine you see the design for a speedboat and a submarine. It's sufficient look at the designs alone to see that one cannot operate submerged underwater for an extended period of time. There is no need for tests to demonstrate that point.

The reason for EDR/antivirus test suites is because most solutions were closed source black magic and some kind of comparison is needed. That isn't as true in this case.

> Let's see what happens with your default RH/SELinux, Ubuntu/Debian & AppArmor, and for shits and giggles OpenBSD, and see how they fare against exploits and vulnerabilities harvested over the years, and make reproducible labs.

This already exists in the form of security advisories, as I said you are just being very lazy. As I said, feel free to check or ask an LLM to do it for you, to ask how many Debian security advisories were mitigated by AppArmor vs how many RedHat says were mitigated by SELinux. OpenBSD doesn't apply since it has no type of similar system.

> I'm not going to scour security advisories to do tit for tat comparisons, I would rather see a well thought approach to this.

Right, as I said you're being lazy. The information exists, you just don't want to put in the effort, you want a nice blog post you can blindly refer to.

> Just need to someone to expend the effort (and I'm not volunteering).

No one will because there is no need. The designs of these systems show what they can and cannot restrict and that is enough. You should put in, at least, a minimum of effort to understand why these abstract systems are not really comparable to antivirus solutions and why the testsuites you think make sense, don't.