Not at all. Memory management for games is something you have to pay close attention to, no matter what's your strategy for dealing with garbage. But this whole GC discussion is missing several years of technology development and best practices.
Besides, it is not guaranteed that you'll succeed with a manual strategy for dealing with garbage. Naive strategies have a really low throughput. They may not stop the world, but will be slow.
In every other case, you're better off using a language whose resource management model doesn't fight you.
Swift is the best of both worlds in this regard.