It's extremely common. It is unclear to me if the mousedown event just kicks off preloading of the page, or actually forces you to navigate to the page. If it's the latter, you are breaking decades of mouse ux changing a click from a mousedown followed by mouseup.
+1 I do those as well. Never really thought about how common it could be, but it is pretty natural for me to drag the mouse and let it up somewhere else to prevent incorrect navigation.
Dragging sidebar links worked fine for me all but once in Safari. Not sure how to tell whether it's actually being used, but I don't notice any problems.
[Edit: I was using a blocker that was blocking it. Now I've used it and can tell it's working, and I still don't notice any problems using it in Safari.]
It’s common enough to be damn inconvenient if the ability just suddenly disappears.
If it happens on all sites at the same time it’s one thing. If it happens on just your site I will just feel like your site is icky (and I might not even know why).