I don't think that's correct. If you do business in another country, you're subject to the laws of that country. More specifically in the GDPR case, if you're doing business with an EU citizen, who has the rights granted in the GDPR, you're obligated to respect those rights or not do business with that EU citizen (or any other).
While this might be the legal reality (I'm no expert), what exactly is the EU's recourse should a US business be found in violation?
If you have a US business, hosted with US servers, using US banks, accepting payment in US dollars, with no intention to operate an office or hire employees in Europe, what's the EU going to do about a GDPR violation? How does it enforce and collect a fine?
Again, I'm no expert, but it doesn't seem to me like the EU can do anything in that circumstance.