I've had uncomfortable experiences. I don't know what would constitute a panic attack exactly. Get some supervision, stay in a safe place away from people. It's a journey. Even the good enlightening trip wasn't really a pleasurable experience: it was scary at times, but worth it.
I have with LSD, a truly terrifying experience. I think it can still be worth trying but things can surely go very wrong with anxiety and hallucinogens.