Shadow ban for everyone that exceeded the rate limit or just the one attacker? As others have said that's shitty for legitimate users that go over the rate limit.
You should only shadow ban manually marked attackers. The one you are sure, very sure that are not legitimate users. This way you can't annoy real customers, as the shadow ban is not automatic and can't trigger on them.