Not at all true, plenty of herbivores out there that doesn't care for humans as nutrition. Plenty of cultures out there living side by side with earth without subjugating it, and it still hasn't killed those people.
For better or worse, not that many people meet hungry carnivores vs "I could go without human for now" carnivores. They're generally kind of risk averse until hunger strikes, then any existing reason/logic (since you claimed "too smart") goes out the window.
But that's because they're either too smart to try or we have altered their genetic makeup by culling anyone who strayed too close.