Many times though, the operator overloading is just a way to hide what's really going on, making it harder for the programmer to have a good grasp of either what the error conditions might be or why there's an expensive operation going on in a particular statement.
Operator overloading is just a function call. Everything you say about overloading an operator could be said about the function call that would replace the overloaded operator. I've never had trouble recognizing that a use of an operator was being done with a non-primitive type, which is the only way the confusion you describe could possibly happen, and the problem you describe has never happened to me while working on C++ code. I don't know why you think it is even plausible.
Operator overloading is useful for a lot more than just math. Overloading * and -> is useful for pointer-like types. Overloading () is useful for function-like types. Overloading [] is useful for collections and overloading ++ and -- is useful for iterators.