Used to bother me too, as a mathematician. Then I learned about linguistics. If enough people use words a certain way, then it's vernacular. Sadly, I think this usage is long accepted.
"x more than y" means "y + x" - a sum. "x% more than y" means "y + x% = y + y×(x÷100)" and "x% less than y" means "y - x% = y - y×(x÷100)".
To me, "x times more than y" means the same as "x times as much as y" only "y×x" and "x times less than y" means "y÷x". I guess this might seem ambiguous if you bracket the expression like "x times (more than) y", but in practice it's bracketed like "x (times more than) y".
My go-to example is "awful". The original meaning is literally to be full of awe. I think you would be hard-pressed to find a speaker today that adheres to that meaning over simply meaning "very bad", which is almost an antonym.
Languages change and are influenced by their use, so you need to pick your battles. If something can be unambiguously understood then you're going to be fighting an uphill battle and it's only a matter of time
It used to bother me too, and some usage still bothers the hell out of me, but there’s a way "exponentially large" works fine.
An exponential change is a change on a log scale. Instead of current+d it’s exp[current+d]. So exponential change makes sense. “Exponentially higher” means on a log scale it’s higher enough to be worth commenting on. It’s mathematically correctly synonymous with an order(s) of magnitude change.
Of course, that gets you into the debate over what exponent or log base to use. Is 2x an order of magnitude increase? 1.1x? 10x? It’s all fuzzy.
And then there are people who just use it as “hella” and that still drives me nuts. “It increased by 10% but that matters so much to me that 10% is exponentially larger!” That kind of usage seems to be catching on too as “exponentially” further enters the lexicon of people who don’t remember what an exponent is. I’ll fight that one forever.
The "exponential" is possibly referring the `n` in scientific notation `×10ⁿ`. IE, it's referring to being larger on a logarithmic rather than linear scale.