[Adverblessly]

IIRC both VLC Player and mpv allow you to set a keyboard shortcut for increasing/decreasing audio-video delay and are capable of streaming directly from YouTube (and I believe any website supported by youtube-dl).

I use mpv and whenever I get audio latency from my headphones* I just adjust in the player.

* - It seems like there's some adaptive algorithm where they start off at 0ms latency but if they experience one hiccup they increase to 250ms-300ms and then another hiccup causes them to jump to 500ms. This is also why a keyboard shortcut that let's you adjust the exact a-v delay value is useful beyond just setting the value once.