Where else would it start if not "you're not allowed to say this word because I don't like it"? Personally, I don't see why that word being banned gets a free pass. Would this proverbial slope start with speech at all? I can't imagine why one would accept one word being banned, then stand up to decry another being banned. What word would that be? Why is it more worthy than any other word?
Historically, it doesn't seem like much of a fallacy to say speech controls become more stringent after being introduced. Indeed, historically, one could remove the word "speech" and still be correct. The control doesn't go away in most cases until power changes hands, and even then it's often meeting the new boss, same as the old boss.