You mean more or less severe than long-term negative effects of sugar or alchocol? And there is greater stigma for non-drinkers than for drinkers in our societies so go figure...
I have friends that believe there is a moral obligation to take psychedelics to expand the mind and increase empathy. That’s a little extreme for me, though.