[chillee]

I don't think we're really disagreeing here. Afaik, hard bounds aren't a well defined mathematical concept, and it's fine to have different ones.

What I initially meant by "hard bounds" was any kind of mathematical proof more rigorous than "well, dropout kinda makes your neural network not rely on one feature so that's why it generalizes".

As for your points, I don't think they're really criticisms of PAC bounds.

I'm not familiar with the first point, but it'd be surprising to me if most PAC bounds had that, considering PAC is a framework and not a specific technique...

Your second point is irrelevant to generalization. You're looking for theory about capacity I think? I think learnability also comes into play.

3rd: that is indeed what the bias variance trade-off would imply. It's also why most classic PAC bounds are vacuous for neural networks.

4th: I think that's a fair assumption to make for any meaningful study of generalization.