There's more to it than that.
Having a garbage collector can be faster even, but would you take a faster program if it cost you memory footprint or predictability?
It just depends on what you're doing.
I'm not really disagreeing with you, performance comparisons between gc and non-gc languages DO tend show otherwise, but consider this: a common GC strategy in C++ is arena allocation. Faster allocations and deallocations but more memory overhead. It's still C++ but a little faster in the right situation.