You can put almost infinitely much effort into optimization. When do you stop? Do you test every user-interaction takes less than 100 ms below which it can be considered good?
It’s not as easy as that. What’s acceptable in test might not be acceptable in your customers conditions. It’s difficult to correctly measure the time for every interaction and humans are extremely bad at telling you how fast something is. Then you also have the problems of regressions that stem from other unrelated changes.
I believe it can be made easy, if you have an easy way to profile your GUI. Test the feature, get a timer. If it took more than 20ms on your killer test machine, it might be too slow for some users. (Adjust your heuristic with the context of your application of course.)