[lvh]

Sure, but two reasons that's not quite optimal and you might not want to do that in practice:

1. This tells you the SP is broken; just using individual keys means that doesn't matter anymore if the SP is broken or not. Individual keys is in your control, fixing the SP much less likely so. And you can just set up a practice of doing it for everything and now it's one less thing to test for.

2. That still requires a bit of testing that's somewhat annoying to set up, which most vendorsec practices don't have time for. It's also only one of dozens of things you need to test for. Ignoring audiences is super common, but a more subtle problem is that you can sign a valid SAML assertion _for the wrong domain_, and now you can sign in as a competitor's staff.

As you hint at, having an SP that'll just self-service accept any random metadata.xml at least gives you a fighting chance :)

[wutwutwutwut]

My question was more related to it being tedious. And now you say it requires a bit of testing which is annoying to set up. Isn't testing this just a matter of changing the audience field to something incorrect and try to sign on? This should take like 2 minutes?

[lvh]

If you just change the audience field, the signature will be invalid, so it might tell you the SP won't accept a bad signature, but it doesn't tell you that the SP would accept a correct-signature-for-wrong-audience assertion. And now we've explored two states in that very big tedium space I mentioned; it still doesn't tell you anything about e.g. canonicalization bugs or cross-domain bugs. Those are much harder to test, because they require your IdP to sign specifically crafted assertions malicious, so you can't test them with your standard Okta install or whatever.

So, sure: you can test this one specific bug by replaying an assertion for a different SP. Or you can make your IdP use new key pairs every time and then you're definitionally immune to the entire bug class forever with every SP. Even if replaying the SP takes 2 minutes, getting the tester to a place where they can exploit it takes way longer for most companies, so it's much more effective to just eliminate entire classes of bugs via policy.

TL;DR: you're right (modulo the amount of time) for this particular bug, but why bother? And if you're going to bother testing, why test for this one specific bug that's cheaper to avoid a different way? (I can think of a reason to test; but then the tedium comes in :))