It’s not as easy as that. What’s acceptable in test might not be acceptable in your customers conditions. It’s difficult to correctly measure the time for every interaction and humans are extremely bad at telling you how fast something is. Then you also have the problems of regressions that stem from other unrelated changes.
I believe it can be made easy, if you have an easy way to profile your GUI. Test the feature, get a timer. If it took more than 20ms on your killer test machine, it might be too slow for some users. (Adjust your heuristic with the context of your application of course.)