I think this would be like a firewall or ingress thing that would drop packets that resulted in excess load before they make it to the application server.
[1] https://stripe.com/blog/rate-limiters
You could still overload the service with a sufficiently large attack in either volume of connection requests or number of unique IP addresses.
Token buckets are usually part of an overall resilience strategy rather than a silver bullet to solve all denial of service concerns.
I think this would be like a firewall or ingress thing that would drop packets that resulted in excess load before they make it to the application server.