Game GCs are interesting because you know that the execution is structured like this and you know how much time you have left before you have to switch back to application code for the next frame/time step. There's interesting optimizations you can make around this and could almost completely avoid user-observable GC pauses.
Various GCs can go faster now too. JEP 376 talks about hundreds-of-microsecond work done in pause now that GC no longer has to scan the whole stack.
That said: 1ms? 1ms is getting into the sorts of latency the OS and hardware impose on your program no matter what it does. For example, on x86, a SMI can take 300us, or 1000us if you're unlucky. I've seen softirqs for shitty wifi chips take a hundred milliseconds! And God help you if you take a hard page fault:
You're worried about 1ms latencies, right? So you're mlock()ing all memory? Running RT threads pinned to cores? Carefully using PI and static priorities to avoid inversions? Avoiding blocking IO everywhere, not even for graphics page-flipping? Managing thermal headroom to avoid involuntary clock collapses? And it should go without saying, but I have to ask: you're running a PREEMPT_RT kernel, right?
No? You're not doing any of these things? Then why are you worried about 1ms in GC?