You get used to it (playing a sport hungover, even). If you’re not a competitive athlete on a high level (it’s just a hobby), you’re missing out by not drinking because of your sport. If you enjoy drinking, that is.
Plenty of people compete as a hobby. You might also say if they were drinking, they'd be missing out on performance. If they enjoy performing better, that is.