[nawariata]

This is expected behavior. Browsers follow the link on mouse up event, if you try to drag it away then mouse up happens outside of the button area and link won't be followed. Button remains active as last clicked element, this behavior is also consistent with operating systems.

[gvkv]

Not really. While I agree that the link should not be followed, the button should still return to its normal pre-mouse-down, pre-hover state. Only the focus should remain with the button.