Why? Disrupting a speaker to the level that prevents them speaking (or prevents listening) is an action that clearly infringes on rights of others and restricts freedom of speech. Regardless of content.
Speak at the same time, but in a slightly different place? If you really care about your speech (and not silencing the speech of others), that is clearly acceptable.
I'm not silencing you. We both have the freedom to speak at the same time, in the same place. Such as a political speaker and opposing protesters. Both have the freedom.