I'm curious which specific paragraphs of the Communist Manifesto is it you consider authoritarian? It's a very short text, and only chapter 2 really makes any argument of what Marx and Engels saw as the goal of communist movements, so I presume it must be something in chapter 2 you consider authoritarian.
There are certainly plenty of things in there that implies too much concentration in the power of the state (e.g. monopoly on communication and banking) for someone of more liberal or libertarian (whether left or right) bent if one considers a modern interpretation of the state. Which is why even many Marxists, especially libertarians, look to works like Marx writings on the Paris Commune for more specific details.
As it's still utilized to justify mass genocide by the state of its own citizens, and has happened multiple times and in greater numbers, let's just count this as the more dangerous book.
There are certainly plenty of things in there that implies too much concentration in the power of the state (e.g. monopoly on communication and banking) for someone of more liberal or libertarian (whether left or right) bent if one considers a modern interpretation of the state. Which is why even many Marxists, especially libertarians, look to works like Marx writings on the Paris Commune for more specific details.