Yeah that's an interesting problem. I mean the simple answer is you just make an unsubscribe method. But that doesn't solve the layer of adding and removing subscription handlers.
Using a subtree, childlist MutationObserver, you can implement a disconnectedCallback for non-custom elements. In the callback, you can clean-up any event handlers that you have added to the element yourself.