In absolute numbers you're right because millions of people are tourists compared to a few wealthy space tourists. But that's a strange metric to use, the space tourists are part of the regular tourist group as well. If we compare energy usage per tourist the numbers will be stacked against space tourism.
Regular tourism is bad, but rocket tourism is a new kind of bad because it puts pollutants in the upper atmosphere and we really don't know what the effects of that will be.