Devil's advocate here. Lots people say the same thing about religion. In fact, you could almost replace meditation with religion in your comment. Quite illuminating.
And it's probably true for religion too. That doesn't constitute a counterargument. The negative aspects of religion (for example, you can argue that religious faith is exclusionary and confusing, and so on) don't carry over to meditation just because they have something in common.