Money that is not required to be paid cannot be called a tax. Taxes are not voluntary. Things like tax avoidance don't even exist, by definition of the word.
What would you call going out of your way to buy cigarettes in a lower tax jurisdiction? The word avoidance seems pretty fitting here. Taxes are as voluntary as the activity being taxed
VAT is a consumption tax. If you consume goods in a place with lower consumption taxes then the tax is doing exactly what it is meant to do.
If you buy consumer goods in a low-VAT place but transport them to a high-VAT place and consume them, then that is clearly illegitimate for the same reason that it is legitimate for VAT to be applied to imports and refunded to exporters.
Within EU, you have free movement of goods, and the VAT is generally paid in the country of sale, not in the country of of the consumer. There are a few exceptions to this, and other regions might have other trade agreements.
Within the EU definition, VAT taxes the company making the sale for the increase in value of a product through the processes the companies and distributors applied to it, and is not a taxation of the receipt of said value.
There is nothing illegitimate about then consuming the product in a different country. It just doesn't happen to send money to your own country's treasury.