[smarterclayton]

I'm referring mostly to users who are legitimately able to run workloads on a non-zero percentage of the cluster. It's fairly hard to defend against a user with the ability to schedule even a limited amount of workloads, because good scheduling usually spreads (in which case you can compromise more machines and then take them out of service briefly) or packs (in which case you get put on nice target rich environments).

That's partially why we (openshift) just don't allow containers to run as root by default - it sucks for new users (most images assume root) but it dramatically lowers the risk of compromise across the board in any scenario with multi-tenancy. Agree that with isolated nodes per tenant you wouldn't have this issue, which is a more straightforward RBAC story.