[palosanto]

Are you missing out on something? Yes.

Is there "little downside"? Not necessarily. Compared to other drugs, sure. You're not going to become addicted and it's not going to make your teeth fall out. But let's be real. This is the most potent psychoactive substance ever made by many orders of magnitude. It can connect you to parts of yourself that are deeply buried in your subconscious. This is what brings about the blissful feelings of transcendence, wholeness, oneness, etc. It can also be absolutely terrifying to confront your subconscious, especially if 1) you're not in a good place psychologically and 2) you're not in the presence of someone you trust. What happens is you start to experience fear as the scary things come up or as you start to experience ego death (http://en.wikipedia.org/wiki/Ego_death). You don't realize that the fear is all in your head and you start to project it out onto the external world with paranoid thoughts. Kind of like you're having a nightmare, except that it's happening while you're conscious. This is known as a bad trip. It can leave lasting psychological damage if you don't know how to process the experience.

Another potential downside is that an LSD trip could trigger a midlife (or quarter-life) crisis. You might come back from the transcendental experience and realize that your life is meaningless, that all of your friends are assholes who don't really know you, etc. You might be inspired to make some big changes. From the perspective of your asshole friends, you will have changed "and not in a good way" ;) But this short-term downside is an upside if you take the long view. If you get your midlife crisis out of the way in your twenties, you can spend the rest of your life doing something meaningful.

There's no way to experience the transcendental upside without the potential downside. But if you're willing to face your own demons, and you take precautions by having a trusted + sober guide, you'll be fine.