That is the lowest possible bar for free speech, which as a philosophical principle goes way further. Here's a good talk from Hitchens on the topic: https://www.youtube.com/watch?v=4Z2uzEM0ugY
I'm not a free speech person, but since I think you're unwittingly falling for a strawman I think I can help.